SEO for an Amazon-Sized Site: The Real Playbook

Say you’re handed SEO responsibility for a site with Amazon’s scale — hundreds of millions of pages, thousands of SKUs changing daily, multiple markets. What does that job actually look like?

It’s worth being precise about one thing first: Amazon also runs a separate internal ranking engine for its own search bar (amazon.com/s) — the system sellers try to influence with backend keywords, pricing, and reviews. That’s a different job, aimed at a different audience, and it’s not what this piece covers. This is about the job of getting a catalog that size to rank on Google — which turns out to be less about individual pages and almost entirely about architecture. Below a few thousand URLs, SEO is content and links. Past that threshold, four structural problems take over: crawl budget, duplicate content, faceted navigation, and indexation velocity. Everything else in this piece builds on those four.

1. Crawl Budget: Spending Google’s Limited Attention Correctly

Crawl budget is the finite amount of crawling Google allocates to your site in a given window — a function of how fast your server can handle requests and how much Google actually wants to crawl you, based on your site’s perceived value (Crawlvision). It’s meaningless for a small site; Google crawls everything easily. It becomes the central constraint once you’re running tens of thousands of SKUs, because bots will spend that budget somewhere — and if it’s spent on low-value URLs, your real products don’t get discovered or refreshed.

One documented case: an apparel marketplace running 500,000 SKUs exposed 50 million URLs through parameterized faceted navigation. Log files showed Googlebot spending 72% of its crawl activity on filtered pages that generated negligible clicks (Venue Cloud). After the team added a parameter whitelist and canonicalized filtered URLs back to clean category pages, non-canonical crawl hits dropped 60% in three weeks, and indexed products rose 14%. That’s the actual size of the win available here — not a marginal tweak.

The audit that matters: pull server logs, not just Search Console. If a large share of bot hits land on parameter or filter URLs instead of product/category pages, that’s the highest-leverage fix available, before you touch a single piece of content.

2. Faceted Navigation: Deciding What Deserves a URL

Filters — brand, size, color, price — are useful to shoppers and dangerous to crawlers, because every combination is a technically distinct, crawlable URL. A catalog with 10,000 products and six filter dimensions can generate millions of combinatorial pages, almost none of which deserve to be indexed (JourneyH). Left unmanaged, this wastes crawl budget and creates keyword cannibalization — multiple URLs competing for the same query instead of one page collecting all the ranking signal.

The fix isn’t “block everything” or “index everything” — it’s a per-facet decision, made deliberately:

• Canonicalize low-value filter combinations back to the clean category page
• noindex, follow combinations that are useful for users but not worth ranking independently
• Block via robots.txt or URL parameter handling for combinations that shouldn’t be crawled at all

The one trap worth naming: blocking too aggressively costs you real long-tail traffic. A high-demand combination — “waterproof hiking boots size 10” — might deserve its own indexable page even though “size 10” alone never should. This is a judgment call per facet, not a blanket policy.

3. Duplicate & Near-Duplicate Content: Consolidating Signal

On any large catalog, the same product can end up represented on multiple URLs — vendor variants, accidental re-listings, near-identical pages generated by a templating system. Without a clear canonical policy, search engines get conflicting signals about which version to index, and the ranking authority that should accumulate on one page splits across several — none of which performs as well as the consolidated version would.

What this looks like when it goes wrong, concretely: search “Lego 71016” on Amazon right now, and you’ll find two separate listings for the identical set — same product, same UPC — with reviews and sales history split across both instead of consolidated on one page (seller forum thread). It happens two ways: accidentally, when a new listing is created without checking an existing UPC match, and deliberately, when a competitor clones a listing under a new ID specifically to dodge shared competition on the original page — a policy violation Amazon enforces reactively, mostly after a rights-holder complaint, rather than catching upfront (Amalytix). Since signals like reviews and conversion rate accrue per-page, neither duplicate ever reaches the threshold the combined page would hit. The split isn’t cosmetic — it’s actively rank-suppressing.

The audit that matters: map every product with more than one live URL, and check whether your canonical mechanism — a rel=canonical tag, a parent-child variant structure, or a manual merge workflow — is actually enforced in practice. Having the tooling isn’t the same as it being applied consistently at scale.

4. Rendering Budget: The Cost Hiding Inside Crawl Budget

This one’s underweighted in most technical SEO conversations, and it compounds everything above. Google’s indexing pipeline runs in two phases: first it fetches raw HTML, then — separately, sometimes hours or days later — it queues the page for rendering, where a headless Chromium instance executes your JavaScript and captures the real DOM (Born Digital). If your product pages depend on client-side JavaScript to render price, availability, or even core content, Google may be indexing a near-empty page for an unpredictable stretch of time before the render queue catches up.

The rendering queue itself prioritizes by site authority — a page from a well-linked, high-traffic domain gets rendered faster than one from a low-authority site, and every AJAX call your page makes during that render counts against crawl budget too (EdgeComet). At Amazon’s scale, that queue effect multiplies: thousands of templated pages competing for rendering priority, not just crawl priority. Server-side rendering or static generation for anything revenue-critical (price, stock status, title) removes this risk entirely — client-side rendering should be reserved for interactive UI, not the content that needs to be indexed and ranked.

5. Indexation Velocity: How Fast New Inventory Becomes Visible

Time-to-index is its own metric, separate from whether a page is capable of ranking once found. Google’s “Discovered – currently not indexed” status is the clearest symptom of this breaking down: Google has found the URL but hasn’t prioritized crawling it, often because of server load limits or a page pattern that looks like a near-duplicate of something already indexed (ZipTie.dev). At real scale, indexed pages can also silently drop back out over time as Google deprioritizes low-value duplicates — one first-party analysis of a million-URL site found 16% of pages had reverted to effectively unindexed status without the site owner noticing through standard Search Console reporting (Indexing Insight).

Log files are the actual diagnostic tool here — and this deserves to be said plainly: Search Console is not enough. Search Console shows aggregated, sampled, Google-only data with a reporting delay. Server logs are ground truth — every request, every bot, every status code, timestamped, no sampling (LinkGraph; Search Engine Land). Logs answer questions Search Console structurally can’t: is Googlebot actually reaching your new SKUs, how often, and is crawl frequency to your top category pages rising or falling after an architecture change. On sites this size, a five-priority log review — HTTP-code distribution, crawled-vs-uncrawled pages, orphan URLs, crawl frequency by page type, and crawl waste on non-strategic URLs — is the standard advanced audit (Gemeos Agency). In 2026, that same log data increasingly needs to be read for AI crawlers too — GPTBot, ClaudeBot, PerplexityBot all show up in server logs as distinct request patterns, separate from Googlebot (EdgeComet) — a layer that barely existed three years ago and now has to be part of the same review.

For the fix: segment sitemaps by content type and freshness — new/updated inventory in its own file with accurate lastmod — rather than mixing everything into one giant sitemap, which is the single most common reason large catalogs report slow indexation.

6. Structured Data at Scale: Where Small Errors Compound Silently

Product schema (JSON-LD) makes a page eligible for rich results — star ratings, price, availability shown directly in the SERP — and pages with valid structured data see meaningfully higher click-through rates as a result (Digital Applied). It’s not a ranking factor by itself, but it’s a real CTR lever, and at scale it fails in a specific, quiet way: not through missing schema, but through mismatches. Markup says $59, the visible page (updated faster by a promotions system) says $49 — Google treats that conflict as untrustworthy and can drop eligibility entirely (Venue Cloud). Breadcrumb schema is a common casualty too: the visible page shows one category path while the JSON-LD, generated from a different internal taxonomy, shows another.

None of this shows up as a dramatic outage — it shows up as slowly eroding rich-result eligibility across templates, one inconsistency at a time, invisible until someone checks the Enhancements report in Search Console. The audit that matters: don’t validate one happy-path product page and call it done. Test edge cases specifically — out-of-stock items, sale pricing, products with zero reviews — since those are exactly where feed data and page data are most likely to drift apart at scale.

7. Internal Linking Architecture: Routing Authority Deliberately

At small scale, internal linking happens naturally through navigation. At large scale, it has to be designed, because link equity doesn’t distribute itself evenly across millions of pages — it flows along whatever paths actually exist, and if those paths are accidental, so is your ranking distribution. Log analysis routinely surfaces this exact failure: Googlebot crawling a category index heavily while barely touching the individual product pages beneath it, which usually traces back to weak internal linking or a UI pattern that buries deeper pages from both bots and users (Search Engine Land).

The deliberate version of this at scale means treating category hubs, breadcrumbs, and “related product” modules as ranking infrastructure, not just navigation — hub pages that consolidate links to their most important children, breadcrumb trails that match the actual indexed taxonomy (not a legacy one), and cross-links between related SKUs that create paths to lower-authority pages instead of leaving them orphaned.

8. Handling Dead & Discontinued Inventory

Every large catalog loses products constantly — discontinued lines, delisted SKUs, out-of-stock items that may or may not return. The standard playbook: 301-redirect to a genuine replacement or the most relevant category if one exists, serve a real 404/410 if nothing does, or keep the page live with an out-of-stock state and alternatives if the item might come back. What kills crawl efficiency is letting dead URLs sit unresolved — one large-catalog audit found roughly 18% of crawled pages returning broken 404/500 responses, quietly consuming crawl budget that should go to live inventory (ThinkShaw).

Worth noting as a cautionary contrast rather than a model to copy: Amazon’s own removed-listing behavior deviates from this playbook. A delisted ASIN can sit in a “Currently Unavailable” limbo state — page intact, no buy option — for up to three months before full removal, and there’s no redirect mechanism to pass accumulated signal to a replacement page. That’s a worse outcome than the standard approach above, and a good illustration of why consolidation policy has to be decided upfront, not improvised per incident.

9. International Architecture: Sharing or Siloing Authority

Most large sites operating across markets use hreflang tags to link language/region variants of the same domain, letting authority and crawl efficiency share across the cluster rather than starting from zero in each market. This is the standard approach, and it’s the one worth defaulting to for a site you’re building rather than inheriting.

Worth knowing as the exception: Amazon doesn’t do this. Each marketplace — US, UK, DE, IN — ranks entirely independently with zero signal transfer between them; a #1-ranking listing in the US starts from zero the moment it’s listed in Germany (beBOLD Digital). That’s a deliberate siloed-catalog model, not a limitation — but it’s the opposite of what hreflang architecture is designed to achieve, and not something to replicate unless your business genuinely requires fully independent regional catalogs.

Where This Is Heading

The crawler landscape is no longer just Googlebot. GPTBot, ClaudeBot, PerplexityBot, and Amazonbot itself now show up in server logs as distinct, fast-growing patterns, and each one requires a separate decision in robots.txt — blocking GPTBot opts out of AI training data, blocking OAI-SearchBot removes you from ChatGPT’s search results, blocking Googlebot removes you from Google’s index entirely. Same access log, three different strategic calls (Digital Applied). None of this replaces the fundamentals in this piece — it adds a layer on top of them. A catalog with clean canonicalization, sane crawl budget allocation, and reliable indexation is also the catalog best positioned to be read correctly by AI crawlers. The architecture work is still the foundation; it just now has more tenants relying on it.