The Impact of Agentic Traffic on the Web

Some of the content of this article was initially published as a series of articles that I authored for Akamai Technologies, and has been refreshed with additional insight and perspective.

AI bots disrupt the established mode of interaction

2025 was the year of AI bots, agentic AI, and the emerging field of agentic commerce. The bot management industry has discussed the topic at length. AI platforms offer a new way for users to interact with the internet, promising to make consumers’ lives easier by helping them find information faster and streamline their shopping experience. When discussing AI bots, the discussion typically centers on Perplexity, ChatGPT, Gemini, Claude, and other well-known AI platforms. According to Akamai, traffic from AI platforms collecting data for model training and from agents collecting information in real time to complete user tasks increased by more than 200% in 2025. As more startups and established companies jump onto the AI bandwagon and develop agents to assist internet users with various tasks, one can expect traffic trends to continue to rise. To further encourage the adoption of AI agents, a new generation of web browsers is emerging, with Google Chrome introducing AI assistance (Gemini) into its browser and Perplexity and OpenAI following in Google's footsteps by introducing Comet and Atlas, respectively, both based on Chromium. This new type of interaction disrupts established models of online communication and bot management strategies.

Before we take this discussion any further, let’s define a few components from this new AI world. First, there are AI platforms that run Large Language Models (LLMs), such as Gemini from Google, ChatGPT from OpenAI, Copilot from Microsoft, Claude from Perplexity, and several others. To function, LLMs must be trained on large amounts of data collected by scraping websites using bots. LLMs are designed to answer questions such as “What are the most interesting things to see in Oregon, and what are the best places to stay?” However, LLMs alone cannot complete complex tasks such as searching for an itinerary, booking a flight, and reserving a rental car and hotel rooms. This is where AI agents come in. According to Anthopic, "Agent can be defined [...] as fully autonomous systems that operate independently over extended periods, using various tools to accomplish complex tasks.” Agents may be autonomous in determining how to execute and solve a task. Still, it is essential to note that the task is always initiated by human interaction with the AI platform. To complete a complex task, an agent may consult an LLM or other agents. AI agents are a new class of “good bots” that bot management products must learn to identify and categorize accurately. They often use headless browser technology to render and ingest collected content, or to execute actions on the target site on behalf of users. They are not designed with malicious intent. However, this doesn’t mean that a user cannot trick them into performing malicious activity. Guardrails must be implemented within the AI platform to prevent such issues; however, this topic is beyond the scope of this paper.

Figure 1 illustrates the interaction among users, AI agents, LLMs, and websites. MCP (Model Context Protocol) servers may proxy a website to assist an AI agent in identifying the appropriate API to complete a task, such as retrieving a flight schedule or booking a seat. Without the MCP server, AI agents must locate the relevant web API themselves to complete a task.

Fig. 1: User interaction with a website through AI agents.

This more complex interaction involving AI agents disrupts established protocols and internet revenue models. Figuring out how to deal with AI bots is becoming an existential problem for publishers. For merchants, AI bots don’t pose an imminent threat but rather represent an opportunity.

Impact on e-commerce

Despite initial market wariness about LLM platforms and AI agents, and a knee-jerk reaction to block this new type of automated (bot) activity that collects massive amounts of data, e-commerce website owners have quickly reversed course and are now seeking to optimize their infrastructure to accommodate AI agents.

The travel industry has so far reacted positively to agentic traffic. The adoption of using an AI agent to book a trip may vary depending on whether you’re booking a business trip or a vacation. I could see myself asking ChatGPT to book my airfare and hotels for a recurring business trip to a particular destination, where I usually take more or less the same flight and stay at the same hotel. However, I don’t know if I could trust ChatGPT for a more complex task like booking a family vacation for me, since in this case, it’s not just about the overall cost and the destination, but it’s also about discovering the destination and figuring out the itinerary, possible activities, and points of interest to see. In other words, I may use ChatGPT to get a general idea of my destination, but I don’t think I can trust AI to know what I’m looking for and book the right hotels in a location that we would enjoy. A complex task like this also requires providing highly detailed descriptions to ensure that the agent makes the appropriate choices. 

For retailers, introducing an AI application to offer a personal shopper assistant experience makes sense, provided you know exactly what you’re looking for. Buying groceries seems like a straightforward proposition for agentic commerce, but shopping for other items may be a different story. Shoppers like me generally only have a vague idea of what they want until they see a specific item. In some cases, describing what they want to the AI system may be challenging. However, one can imagine an AI agent that serves as a personal shopper, asking the user about their tastes, offering different product options for the user to select, and, through trial and error, learning the user's preferences.

On the surface, agentic commerce doesn’t seem to be a problem for e-commerce and may help attract more customers to a brand. However, in a purely agentic interaction, merchants may miss opportunities for upselling. It is true that if one is interested in buying a specific item, AI bots can select products or services that match the criteria and handle the checkout process. However, through the “agentic channel,” end users will not see the recommended products and services that the website’s marketing team goes to great lengths to position, thereby potentially reducing the dollar amount a user spends. 

Now, when examining hype-event sales — such as limited-edition sneaker drops, concert tickets, Pokémon card restocks, or Labubu doll releases — AI agents pose a new challenge. For years, custom bots have been developed to automate the purchase of limited-availability items, a practice the industry has fiercely opposed to prevent scalping and price inflation while also ensuring consumer fairness. AI agents can now easily perform all the tasks that custom bots were previously designed to handle. Traditional bots may soon be replaced by AI agents that monitor the release of specific items and purchase them when available. This will require retailers to adopt different policies regarding AI agents for products with limited availability.

Impact on the publishers and public forums

Search engines such as Google now provide an AI summary along with the traditional links to sites relevant to the search query, which may be sufficient for most users who don’t necessarily want to delve deeply into a topic. I admit that these summaries are convenient, and the information returned is often enough for me. This new behavior poses a problem for publishers, new outlets, and online forums that provide only information and rely on online advertising and affiliate marketing to generate revenue, because studies show that the number of visits to the sites where the original data were collected to obtain the answer is dropping. If one wants to learn about the latest news or tomorrow’s local weather, they may instead ask ChatGPT or similar platforms for highlights rather than visiting CNN or the Weather Channel websites. Similarly, as an engineer, I can be satisfied with the code example provided by Claude at Anthropic and no longer need to consult forums such as Stack Overflow. This drop in page views reduces revenue for sites that depend on online advertising and, for publishers, subscriptions. Online advertising and, to some extent, the collection of user data have enabled a significant portion of the internet to remain free and sustain publishers. The new mode of interaction through AI agents poses a threat to this business model.

Impact on the banking and insurance industry

Banks and insurance companies are also trying to understand how to adapt to AI agents. After all, one may use them to look for an insurance provider after purchasing a car or a house. Similarly, one may use agents to shop around for the best mortgage rate or credit card offers. So ultimately, the insurance and banking providers will need to consider optimizing access to their sites for AI agents. Today, some companies act as brokers to compare insurance policies. Geyco in the US is an example in the insurance world, and update.com focuses on the credit card offers. Banks need to collect substantial information before providing a credit offer, and using an AI agent would help streamline the process and save significant time for consumers. As consumers continue to adopt AI agents, they may expect the ability to transfer funds between accounts or pay bills through these agents. This industry is heavily regulated and typically lags in adopting new technologies. However, for a startup building an agent to customize credit card, mortgage, or insurance offerings based on consumers' criteria and personal situations, this could be a promising opportunity and a new source of business for the industry. This brings AI agents into the PII domain, and the industry will want to ensure that its customers' information is secure during such interactions. However, failing to optimize for AI could result in a missed opportunity.

Conclusion

LLM and AI agents have been available for a few years, but in 2025, adoption of these systems for interacting with websites increased significantly. In Part 1, we defined the basic architecture of AI platforms and how their usage affects various industries. See part two, where we review emerging protocols to facilitate interaction between agents and agentic commerce, and to provide the publishing industry with possible solutions to regain control over their revenue and the use of their content.