Agentic Web 2027: When AI Agents Visit Your Website

Imagine a potential customer tells their ChatGPT agent: “Find me an electrician in Göttingen available next week, collect three quotes, and book the cheapest one with four or more stars.” Five minutes later, the agent reports back: appointment booked, confirmation in the inbox. The user never opened a single website themselves.

This isn’t science fiction. ChatGPT Agent has been generally available since July 2025. Claude Computer Use has been in beta since October 2024. Perplexity launched “Buy with Pro” as early as November 2024. And Anthropic transferred the Model Context Protocol (MCP) to the newly founded Agentic AI Foundation under the Linux Foundation at the end of 2025 — co-funded by OpenAI, Block, and Anthropic itself. Over 10,000 public MCP servers are already active.

After the GEO era — where the goal was getting cited by AI search engines — the next phase begins: the agentic web. Here, AI doesn’t just read your page, it acts on it. It fills forms, requests quotes, books appointments, closes deals. This article shows what that concretely means for SMB websites — and which seven measures you should prepare today.

What Is the Agentic Web?

The classic web was built for humans. The user types a URL, reads text, clicks buttons, fills forms. The agentic web adds a second layer: autonomous AI systems that perform the same actions on behalf of users.

An agent isn’t a chatbot. A chatbot answers questions. An agent pursues a goal across multiple steps: research, compare, decide, act. It opens your website, interprets the content, navigates menus, clicks links, fills input fields, and submits requests — without a human stepping in.

The difference from classic web scraping: agents operate goal-driven and contextually. They don’t follow a prescribed sequence; they adapt their steps in real time based on what they see on the page. This is possible because modern multimodal models — Claude 3.5 Sonnet, GPT-5, Gemini 2.5 — can understand screenshots of websites and derive actions from them.

The Scale: Numbers That Show the Direction

Adoption is moving faster than any previous web technology:

• 45% of shoppers already use AI shopping agents for purchase decisions, according to a study by Ekamoira
• 900 million weekly users use ChatGPT with active web access (as of 2026)
• 1,445% more inquiries about multi-agent systems were recorded by Gartner from Q1 2024 to Q2 2025
• McKinsey forecasts $3–5 trillion in agentic commerce volume by 2030

This isn’t a fringe phenomenon. This is the next infrastructure shift — comparable to the mobile-web transition around 2013.

The Five Types of AI Agents Visiting Your Website

Not every agent behaves the same way. For technical and strategic preparation, it’s important to distinguish the types:

Agent	Origin	What it does	Your action
ChatGPT Agent	OpenAI	Browses, clicks, fills forms, shops	Structured content, clear CTAs
Claude Computer Use	Anthropic	Controls browser like a human via screenshot	Visual hierarchy, readable UI
Perplexity Actions	Perplexity	Researches + acts (e.g., bookings)	Verifiable facts, clear pricing
Google Gemini with Project Mariner	Google	Web navigation directly from browser	Schema markup, semantic HTML
MCP-based custom agents	Companies / developers	Specialized workflows via protocol	MCP endpoint (optional)

The crucial point: all five types interpret your website like a human, but with different strengths and weaknesses. An AI reads HTML structure faster but recognizes hidden CSS tricks less well. It understands tables better than flowing text. And it gives up on poorly labeled buttons.

Agentic Commerce: When AI Shops

The most advanced use case is commerce. Three competing protocols are already in production:

OpenAI ACP (Agentic Commerce Protocol). Integrated into ChatGPT as “Instant Checkout.” Live since September 2025, with Stripe handling settlement. OpenAI pulled the first iteration in March 2026 after only about 30 merchants were active — product data came from web scraping and was too inaccurate. The second generation now runs on structured product feeds.

Google UCP (Unified Commerce Protocol). Partnership with Walmart, Target, Shopify, Etsy, and over 20 additional retailers. Focus: intent-based discovery — “I need winter work boots, size 44, under 150 euros” leads directly to matching products with a purchase option.

Shopify Storefront MCP. Live since early 2026. Hydrogen stores on Oxygen can now embed custom AI agents directly into the storefront. That means your Shopify store can talk to ChatGPT, Claude, and Perplexity — not just deliver data, but run ordering processes.

For SMBs this means: anyone running a Shopify store today has already laid the foundation for agentic commerce. Anyone on a custom solution should check whether product data is available in a structured format (Google Merchant feed, Schema.org Product markup, open API).

What Happens If You Don’t Participate?

Nothing — and that’s exactly the problem. An agent that doesn’t understand your website ignores you. It picks the competitor where the checkout works, product data is clean, and prices are clear. Visibility in classic Google search doesn’t protect you: if the user never lands on Google but goes directly to the agent, you lose access to the customer.

MCP: The Protocol Changing the Web

Model Context Protocol is what TCP/IP was for the internet: an open standard that creates a common language. MCP defines how AI systems communicate with external tools, data sources, and websites — standardized, authenticated, auditable.

For website operators, MCP opens three options:

Option 1: Do nothing — remain passively reachable. The simplest strategy. Agents access your site via normal web browsing. Your job: build the site so it’s understandable for an agent. For 90% of SMB websites, this is the right choice.

Option 2: Expose public data as an MCP server. If your website has structured information (price list, availability, product catalog, knowledge base), you can offer your own MCP server. Agents then retrieve the data directly instead of scraping the page. Faster, more accurate, more energy-efficient.

Option 3: Allow actions via MCP. The most advanced stage. Your MCP endpoint lets agents not only read data but also perform actions — book appointments, request quotes, place orders. This is technically demanding and requires thoughtful authentication, but opens new conversion channels.

{
  "mcpServers": {
    "paulmill-availability": {
      "command": "npx",
      "args": ["-y", "@paulmill/mcp-server"],
      "env": { "API_KEY": "..." }
    }
  }
}

Practically speaking: anyone using a calendar booking link like Cal.com or Calendly today effectively already has an MCP-compatible endpoint — integration is just a matter of the standard.

The Seven Measures SMBs Should Prepare Now

None of these measures requires a complete rebuild. It’s about consistently implementing existing best practices — and thinking one step further in a few places.

1. Semantic HTML Instead of Div Soup

An agent that interprets your page via screenshot relies on visual hierarchy. An agent that reads the DOM needs correct HTML elements. <nav>, <main>, <article>, <button>, <form> — not generic <div> containers with click handlers. This is the same requirement that underpins the BFSG checklist — accessibility and agentic readiness are two sides of the same coin.

2. Expand Schema Markup

Structured data is to agents what road signs are to cars. Particularly relevant:

• LocalBusiness with opening hours, address, contact
• Product with price, availability, images
• Service with description and price range
• FAQPage for frequent questions
• Offer with conditions

More on this in the GEO article — the principles overlap by 80%.

3. Clear, Unambiguous CTA Buttons

“Book now,” “Request a quote,” “Add to cart” — not “Learn more” or “Continue.” An agent meant to execute an action must be able to clearly map the button text to an intention. Don’t hide actions behind icons without aria-label. A naked arrow-icon button is invisible to an agent.

4. Make Pricing Transparent

Nothing frustrates an agent — and the user behind it — like “price on request.” If your services aren’t flat-rate, at least offer a range, package prices, or a calculator. My cost calculator, for example, gives an agent a usable answer without having to fill out a contact form.

5. Simplify Forms

Every required field, every CAPTCHA, every multi-step wizard is a hurdle — for humans and agents alike. Reduce contact forms to the minimum: name, email, message. Everything else can come in a second step.

Particularly important: no Cloudflare Turnstile or reCAPTCHA challenges that actively lock out agents. If you need spam protection, use honeypot fields or server-side rate limiting — both invisible to agents.

6. Render Page Logic Server-Side

Websites that only load content via JavaScript are only partially readable for agents. Most agents wait a maximum of 3–5 seconds for hydration. Whoever renders server-side — like this website with Astro — is immediately parseable.

Performance matters double in the agentic web: an agent comparing ten websites will abandon slow ones. The fundamentals in the Core Web Vitals practice guide still apply.

7. Set Up llms.txt and agents.txt

llms.txt is the existing standard for AI systems and is still supported by new providers in 2026. Complementing it, agents.txt is emerging as a possible new standard — a file that explains to agents which actions are allowed on the site, which MCP endpoints exist, and how rate limits work.

Anyone with a clean llms.txt today is well prepared for the transition. You’ll find an example in the GEO article.

Agentic Design: Two Audiences, One Interface

The exciting thing about the agentic web is that you don’t have to choose between human and agent. A website that’s readable for agents is also better for humans: clearer, faster, more accessible.

Three principles that have proven themselves in practice:

Explicit over implicit. An icon alone isn’t enough. A cart icon without text or aria-label is invisible to an agent — and also to any user with low vision. Use text AND icon, not just one of them.

Structure over ornament. A page with five nested animations building one after the other confuses agents that interpret a screenshot. Use animation deliberately, not decoratively. The discussion in Web design trends 2026 on this is still relevant.

Redundancy is fine. A price in a schema markup, in visible text, AND in a <meta> tag isn’t waste — it’s robustness. Different agents extract information via different paths.

What You Should Measure

Classic analytics fall short. An agent visit often doesn’t generate classic session traffic in the Google Analytics sense — but it leads to conversions you’d otherwise miss.

Three KPIs that matter in the agentic web:

Identify agent traffic. Specific user agents appear in server logs: ChatGPT-User, Claude-Web, PerplexityBot, Bytespider. In your log analysis, check what share of your traffic comes from agents. Tools like Cloudflare Analytics and GoatCounter visualize this out of the box.

Referrals from AI sources. Visitors with a referer of chat.openai.com, perplexity.ai, gemini.google.com, or claude.ai show that an agent recommended your site and the user clicked through directly. That’s pure gold.

“Zero-contact” conversion path. Some conversions run without a visible referer — the agent has already convinced the user before they reach your site. Watch for direct traffic with high conversion rates after specific campaigns or content releases.

Frequently Asked Questions About the Agentic Web

Isn’t this all just hype?

No. ChatGPT has 900 million weekly active users. Claude Computer Use runs in production in enterprise environments. MCP has over 10,000 active servers and is being developed further by a nonprofit foundation under the Linux umbrella. These aren’t demo numbers — this is infrastructure.

What is hype: forecasts that classic web will be dead in two years. The reality is hybrid — humans will keep clicking, but a growing share of commerce and information seeking runs through agents.

Do I need to build an MCP server now?

For 90% of SMBs: no. A clean, semantic, fast website with structured content reaches all agents through normal web browsing. MCP servers make sense if you have structured data (product catalog, availability, extensive knowledge base) or complex transactions (appointments, orders with configuration).

What about agents comparing prices?

A risk — and also an opportunity. Whoever offers the lowest price wins automatically. Whoever can’t compete on price must deliver other signals: reviews, references, clear differentiators. Transparent prices lose comparison sprints — opaque prices lose everyone.

How do I block unwanted agents?

Via robots.txt. But be careful: blocking GPTBot means losing visibility in ChatGPT and increasingly, traffic. Cloudflare offers an “AI Bot Blocking” feature under Security → Bots that locks out many agents by default. Check there whether your settings match your strategy.

Will agents replace Google?

Not anytime soon. But the share of searches running through agents will grow. Gartner forecasts a 25% decline in traditional search volume by the end of 2026. A quarter century of Google optimization becomes one building block of a broader strategy — not the sole playing field.

What Remains

The agentic web is the logical continuation of the GEO era: first, AIs learned to cite your content — now they learn to work with it. The good news: anyone who lays solid foundations today (semantic HTML, schema markup, fast pages, transparent information) is automatically prepared. There’s no secret agent optimization that exists alongside user optimization — it’s the same work, just with a second audience in mind.

The seven measures in this article aren’t future music. Semantic HTML, clean CTAs, transparent prices, server-side rendering — all of that is already good craft today. Anyone who consistently implements these points wins in classic search results, in AI Overviews, and in the agentic web simultaneously.

The red thread: Make it easy for the machine to understand you — and you make it easy for every human to use you. This isn’t a trend. It’s the new baseline.

If you want to know how agent-ready your website is today, take a look at my Performance & SEO offering — or use the cost calculator for an initial estimate of your project.