What if everything you know about SEO is about to become obsolete? The era of tracking keyword rankings is over. A new, unpredictable force has taken its place: AI-powered answer engines that care less about where your page lands in search results and more about what your content truly means. As search engines transform from simple word matchers into sophisticated interpreters of intent, the strategies that once guaranteed visibility are being rewritten in real time.
In this article, you’ll discover why the old rules no longer apply, how semantic models and probabilistic AI are reshaping the digital landscape, and what it takes to make your brand stand out when every answer is personalized, every query is unique, and the only constant is change. If you want to thrive in a world where rank tracking is dead, read on because the future of search is already here, and it’s nothing like the past.
Here’s what’s in this article
- Lexical vs semantic search
- Probabilistic thinking
- LLMs and answer engines’ general processing flow
- Essential aspects of AI Mode for marketers to consider
- Ways forward
Rather listen to this content? We’ve created an audio summary for you.
Lexical model vs semantic model
In 2018, Google released BERT (Bidirectional Encoder Representations from Transformers) and officially applied it to search in 2019. The essence of BERT is that it converts words into numbers. However, unlike previous models like Word2Vec, BERT captures not only a word but also the words nearby to understand its meaning. So the word bank in “they robbed the bank” would have a different numerical representation than “they went down to the river bank.”
This mathematical model is the smoking gun that search moved away from a lexical mode (words on a page) to a semantic mode (words on a page mean something). Placing a word a specific number of times on a page no longer matters. Your content needs meaning beyond simple keyword frequency.
Probabilistic thinking
LLMs at their core are probabilistic. An LLM is not smart in the classical sense. It simply considers the most probable next word and outputs it. I’ve heard people liken them to “fancy autocomplete.”
How does it learn what should come next? By looking at a copious amount of text that humans have already produced. This is known as training. Through training on books and the millions of pages on the internet, an LLM can predict, based on probability, what a human might write.
There are a few significant bits to consider from the fact that an LLM is a prediction machine. First, associations with your brand are built primarily off of your website, because your website is only a grain of sand at Praia do Cassino. Second, new ideas and innovations that receive limited coverage can be challenging for an LLM to predict, as it needs more reference material.
LLMs and answer engines’ general processing flow
What is this prompt about?
The first thing an LLM needs to do is understand what the prompt is about. The LLM will classify the prompt into a “work stream” category.
From the Claude system prompt leak, we can see one approach this categorization might take:
Claude intelligently adapts its search approach based on the complexity of the query, dynamically scaling from 0 searches when it can answer using its own knowledge to thorough research with over 5 tool calls for complex queries.
We can also infer from the 2024 patent filed by Google called Search with Stateful Chat (US Patent US20240289407A1) a few categories that AI Mode will use:
Identify a classification of the query. The classification may be one of several candidate classifications. Non-limiting examples of candidate classifications may include, for example, (i) “needs creative text generation,” (ii) “needs creative media generation,” (iii) “can benefit from ambient generative summarization,” (iv) “can benefit from SRP summarization,” (v) “would benefit from suggested next step query,” (vi) “needs clarification,” (vii) “do not interfere,” and so forth.
This classification step has implications for the downstream work that the answer engine will need to do. It also determines whether the system will search external sources when producing an answer. This is key to your SEO strategy because if the LLM already knows the answer, it won’t look for additional sources. Don’t waste resources creating content that the system already knows.
One last note. The classification step is iterative, and the LLM will loop through it at each stage during its development of an answer to your query until it’s ready to present a final response.
Query fan out, RAG, and source identification
Once the LLM has determined what to do with the query, it will pass the work to one or more specialized LLMs (or none of the above). For example, if the initial intake LLM determines that the query “needs creative text generation,” it will pass the query to an LLM that specializes in creating text. However, if the input LLM determines that a search is needed, the following section becomes relevant.
When an external search is required, the LLM will determine whether the original prompt should serve as the primary search query or if additional, alternative search queries should be incorporated. Google calls this query fan out. More specifically, from the Search with Stateful Chat patent, here’s a look at what AI Mode might be doing.
These additional/alternative queries may be, for instance, alternative query suggestions, supplemental queries, rewritten versions of the user’s query, and/or “drill down” queries that are generated using the first LLM, and that are meant to direct the user’s search to responsive content that has increased value to the user relative to what would have been returned based solely on the user’s query.
While AI Mode will bring personalization to the forefront of these queries based on your previous interactions, all of the major LLM companies’ systems will create similar synthetic queries.
I’m using Perplexity here since it’s the easiest to see precisely what the system is doing. Here’s a look at some of the conversations I’ve had while working on this article.
Use Video Enhancing Your Content Strategy with Effective Search Techniques
What you’ll notice here that is different from a classical search engine response is that the system can grab results from multiple queries, mash them together and provide a summary of your entire search journey in a single response.
Essentially, Google and others believe that it can ask a better question to provide an answer to your search and are utilizing LLMs to automate this process. At its core, this is what the Search with Stateful Chat patent aims to address.
When planning a vacation, an individual may first search for a flight, then a hotel, then for dinner reservations, and so forth. The longer a search session, the more likely the individual is to become overwhelmed with too much information and/or to lose track of information that was surfaced during prior turns.
While this past section lends more to classical SEO, the details of what’s happening below the surface are essential, so let’s talk about how the system might choose which Search Response Documents (SRDs; using the language from the Search with Stateful Chat patent as it seems most straightforward to understand) to consider in its response before moving on to other critical components of the system.
What is Retrieval-Augmented Generation (RAG)?
Retrieval-Augmented Generation (RAG) combines generative AI with search capabilities. RAG models retrieve relevant documents from external sources and use that information to generate more accurate, knowledge-rich responses. This improves factual accuracy and extends AI knowledge beyond its training data.
RAG is like a human who understands the English language but needs to conduct a Google search to gain a deeper understanding of a subject or reference census data to get the most up-to-date population information.
When the LLM searches the internet, it examines the results, which are more or less traditional search results, and asks, “What here is relevant?” Relevance can be determined using math. This is good news because machines can understand math. Here are the essential parts of the process:
- Convert words into embeddings (a numerical representation of a word or groups of words as coordinates on a multidimensional map).
- Compare one set of embeddings to another set of embeddings using cosine similarity.
What is cosine similarity?
Cosine similarity measures how similar two vectors (points on a map) are by calculating the cosine of the angle between them. It ranges from -1 to 1, where 1 means identical direction, 0 means no similarity, and -1 means opposite direction.
The closer to 1 the cosine similarity value, the more similar in meaning the two sets of words. In the case of Google, the system isn’t looking for the highest value, but rather that it meets a threshold. This ensures the content is relevant to the query and that the system can now focus on other measures of quality to determine what is included in the RAG results set.
The key point here is that trying to get your content a similarity score of close to 1 will only see diminishing returns. Hit a threshold, but don’t sweat if your content isn’t the highest. When analyzing content on the Search Engine Results Page (SERP), we review the average cosine similarity score of the top ten and assume it is our threshold. So long as our content meets or exceeds that standard, we’re happy.
Quality measures
Back in 2016, a Google engineer famously said: “We don’t understand documents, we’re faking it.” Google has come a long way since then.
As you can imagine, there are numerous ways to determine document quality. Google classifies these factors into three buckets: query-dependent measures, query-independent measures and user-dependent measures; however, they can be summarized into two broad categories:
- Trust signals
- Engagement signals
Trust signals
Trust signals are data points that enable a system to assess the accuracy of a piece of content. While sources with low trust can be accurate, a known trustworthy site has the advantage of a proven history on its side.
Here are some known trust signals:
- Author recognition: How well known is the author or publisher in the area of expertise that the query implies?
- Selection rates: Does the document show up in multiple searches?
- Inbound links: Do other documents link to the source document?
- Freshness: How recent was the document created, or has it been updated recently?
Learn more about Google’s freshness system.
For example, the journal Nature is challenging to get published in because its editorial standards are high. Therefore, the content published in Nature is more likely to be accurate and trustworthy, and the system can be confident in using that publication’s information in its answer. However, LLMs aren’t expected to be trained on the most recent editions of the journal, so if a similar trustworthy publication can be found with more freshness, it will be prioritized.
While we don’t have empirical proof, it’s likely that trust signals are the stronger decision criteria for LLMs and will be used as a tiebreaker between two equally relevant documents.
Engagement signals
While it’s likely LLMs will kill traffic to our websites, engagement metrics still matter. Essentially, engagement can be simplified to whether the content will entice activity. That activity could be reading further, clicking a link or asking additional questions.
Here are some known engagement signals:
- Click-through rates (CTR): How often does the document get clicked when shown?
- Location (user and document): What context does the person’s location have to the query? Does the document’s location satisfy the query?
- Language (user and document): What context does the person’s language have to the query? Does the document’s language satisfy the query?
- Content metadata: What can the LLM infer about the content of the document from its metadata?
Grounding and verification
Before the output of a final answer (cue your best Regis Philbin voice), the LLM will check its answer like a school teacher in June. At this stage, the system considers which documents to create a reference link for and double-checks its work.
There are several ways that an LLM can verify information. First, we need to remember that the system is a machine; therefore, embeddings form the foundation. Based on the Search with Stateful Chat patent, AI Mode may use embeddings to compare its answer to source documents.
For example, the system can compare an embedding of the portion of the NL based summary to an embedding of a portion of the SRD to determine a distance measure (e.g., in embedding space) and determine the SRD verifies the portion of NL based summary if the distance measure satisfies a threshold.
As well, notice that it uses a threshold. Once a document meets a threshold, the system can move to the next step. If the document misses the measure, it’s removed from the set.
Another method of verification can be summed up as a popularity vote. If a document or portion of a document appears in four or more documents, where trustworthy documents are weighted more significantly, the content being verified will be considered acceptable.
For example the confidence measure of a portion can be based on trustworthiness of the SRD(s)) that verify that portion and/or a quantity of the SRD(s) that verify that portion. For instance, the confidence measure can reflect greater confidence when four SRDs verify that portion than when only one SRD verifies that portion. Also, for instance, the confidence measure can reflect greater confidence when four highly trustworthy SRDs verify that portion than when four less trustworthy SRDs verify that portion.
Essential aspects of AI Mode for marketers to consider
Individual and personalized response
Generic results are table stakes now. For an LLM to win, according to the platform playbook, it needs to provide the most responsive answer relevant to you in a particular moment. To get that, a system needs to understand the context of your query and who you are as an individual.
Google will use your previous interactions and other data sources to tailor the LLM to you as an individual. While products like Nest and FitBit are separate companies within the Alphabet ecosystem, it’s not far-fetched to see the company flip a switch and bring those data sets into the conversation. Imagine looking for a pair of new running shoes or an air conditioner, and your “assistant” knows exactly how you’ve used the products in the past, so it can inform you about what you should get in the future.
This section can be summed up in a nutshell. If you and I enter the same prompt at the same time on the same device, we’ll receive two different responses that would be beyond the variability that temperature settings would produce.
Multimodal inputs and outputs
The system can take images, audio and video input. It also trains and sources answers from these multimodal content types. Additionally, advanced systems can output text, images, and video. What this means is that text-based content is now table stakes. If you want to move into a leading space, you need to produce content that goes beyond just text on a website.
SEOs have little control over the final output
While I think traditional SEOs will be able to get their content into the results sets, the final output isn’t up to them; it’s up to the agent that interacts with the human between your site and the query input. If you expect to be able to control the final output from an LLM finely, you’re going to be frustrated.
Ways forward
Are we still doing SEO?
There are numerous aspects of optimizing your content for LLMs. One of the key points to remember is that LLMs utilize a search function to assist in providing answers. When that search function happens, it’s more of a traditional search. However, beyond the technical SEO elements that ensure your content can be crawled, an LLM uses a different approach to select documents for inclusion in its research and eventual answer.
What this means is that you still need to continue doing SEO as you have in the past:
- Prioritize a primary keyword in your page titles and headings
- Build links to and from your content
- Make your content engaging to the visitor
- Structure your content in a logical way
And then you need to layer in new ways of optimizing your content:
- Seek mentions of your brand on other websites with or without a link
- Use embeddings and cosine similarity to measure relevance
- Compare your content to other content at a passage level
- Provide context when possible to enhance the meaning of what you are creating: e.g. a green apple picked from an orchard in Kashmir, India, is better than a green apple.
AEO, AGO, or whatever other three-letter name you want to give it, is simply the next evolution of businesses seeking visibility.
Need to change the way we value our capabilities
While traditional SEO will still have an impact, it’s clear that rank tracking is dead (and has probably been on a respirator for the last couple of years). LLM answer engines are personalized to you, the searcher, and rank trackers will not have access to individual accounts. This means that getting an average rank or volume count is improbable.
Furthermore, the way people utilize LLMs differs. People use longer queries, and it’s highly unusual for two searches to be the same. A person typing in a keyword like [digital marketing] is rare, but they may enter [digital marketing company that helps medium-sized ecommerce businesses].
So, how do we measure our success?
Brand visibility
While we can’t see search results at scale, we can understand how an LLM perceives our brand. LLMs are trained on vast amounts of content and use RAG methods to enhance their understanding. You can use this to your advantage.
Understand your current associations
What associations does the LLM have about your brand and industry? How well established are you? You’ll want to review these questions from both grounded and ungrounded perspectives. It’s also a brilliant idea to measure these associations over time to see if any trends emerge.
Here are the prompts you should use:
Grounded
List ten things that you associate with [brand name]. Rank them in order from highest to lowest association.
List ten things that you associate with [brand name]. Rank them in order from highest to lowest association. Do not search the internet.
Ungrounded
List ten things that you associate with [main category]. Rank them in order from highest to lowest association.
List ten things that you associate with [main category]. Rank them in order from highest to lowest association. Do not search the internet.
In addition to analyzing your current position on an LLM, reviewing your brand using a tool like Also Asked can help you understand how LLMs and consumers see your brand and product.
Total revenue
Ultimately, a business needs to generate revenue. Simply understanding how your actions impact revenue is a good start to succeeding as a marketer. First, start tracking revenue in a dashboard if you’re not already. Then create a way to track the activity and changes you are making. Periodically, look for correlations between your actions and revenue.
Cover topics completely
LLM methods, such as query fan out, mean that a search for [digital marketing consultant] also encompasses dozens of related searches. You need to focus on topic-level domination rather than keyword-level optimization.
One of the best ways to do this is to understand your customer. Figure out what they want at every stage of their journey. Everything, from the early stage where they’re not even aware of a problem, to being problem- and solution-aware, right through to purchase and advocacy. You need to know what questions they are asking at each point in their journey.
Here’s an example using a tourism company as our sample. You’ll need to go much deeper but this should spark some ideas to get you started.
| Buyer journey stage | Sample questions asked |
|---|---|
| Dreaming | Family vacation ideas Beach vacation ideas |
| Considering | Best places to visit in Canada Scenic train journeys in Canada |
| Planning | Best time to visit Toronto 14-day itinerary maritimes |
| Booking | Vacation condo rentals Flights to YYZ |
| Experiencing | Things to do near Toronto Hiking trails in Toronto |
| Sharing | Where to leave a review for [tour company/destination] Best hashtags for travel photos |
You’ll notice that some of these questions could fall into other categories. The buyer journey is not a straight line. People can skip ahead, move back or search the same question later with a different perspective. AI Mode aims to make this process seamless, enabling the system to understand your previous questions, consider future steps, and provide a more comprehensive answer based on that.
Get off your website and focus on external sources
Since AI Mode is probabilistic, the more your brand gets mentioned around semantic and closely related topics (near embeddings), the more you’ll see your brand in responses to the system.
Here are a few ways to make that happen:
Digital PR
Digital PR is a marketing strategy that uses online platforms to build brand awareness and earn high-quality backlinks through media coverage, influencer outreach, and content campaigns. It improves SEO by increasing domain authority and driving traffic from reputable publications.
With digital PR you’re looking to get your brand mentioned in the press or by influential people. This increases the number of places where your brand is mentioned and creates external associations. In addition, many of these placements will be on trustworthy websites, which gives you an added boost.
Link building but not for links
You’ll want to continue building links as you always have but now it’s less about getting the actual link and more about the brand mention. Like digital PR, this creates associations and builds your brand as an entity in your industry.
When building links there are three factors you’ll need to consider:
Relevant content
Does the website where you want to place your link have relevant content compared to what your website is about?
Quality domain
Is the website linking to your property high quality?
Local presence
Where is the linking website geographically hosted?
Getting all three of these aligned is difficult and rare. Prioritize relevance and quality domains as these will have the most impact. Remember you can use cosine similarity to help you find the best page to link.
Other channels
Whether it’s social media, an email list or a Slack community, building presence on other channels is critical. YouTube is a favourite since it’s affiliated with Google and has a massive following. You can embed videos on your website, too. This gives you multiple ways to rank for the same term. Video can be especially powerful for early-stage content, as people love to educate themselves and learn new things by watching a YouTube video.
Wrapping up
As search engines and answer engines evolve, the lines between traditional SEO and AI-driven content optimization have blurred. The shift from lexical to semantic models—driven by innovations like BERT and LLMs—means that search is no longer about keyword frequency, but about genuine meaning, context, and relevance. Marketers must now focus on building brand associations, optimizing for topic coverage, and ensuring their content is recognized as trustworthy and engaging across the web.
Success in this new landscape requires more than on-page optimization. It demands a holistic approach: strengthening your brand’s presence through digital PR, strategic link building for mentions (not just links), and leveraging multiple channels to create a web of semantic relevance. Measuring impact now extends beyond rank tracking to understanding brand visibility, associations, and the direct effect on revenue.
If you’re ready to adapt your marketing strategy for the era of AI-powered search and want expert guidance on building a future-proof digital presence, connect with a digital marketing consultant who understands these shifts. Take the next step and reach out to a digital marketing consultant today.