Crawled, indexed and ranked are three different things

The three stages

Crawling is fetching the page. Indexing is deciding to keep it and working out what it covers. Ranking is choosing which stored pages answer a search.

Each stage can fail on its own. A page missing from search has failed at one of them, and the fix is different for each.

Search Console tells you which one, which is why connecting it is the first real step in any search work.

Most advice you read online mixes them up, which is why so much of it does not work. Somebody with an indexing problem is told to build links, and somebody with a ranking problem is told to submit a sitemap.

Most advice online mixes them up, which is why so much of it does not work. Somebody with an indexing problem gets told to build links, and somebody with a ranking problem gets told to submit a sitemap.

What happens during indexing

The engine reads the content, works out the subject, notes the title and description, and decides whether this page adds anything to what it already holds.

It also picks one address as the canonical version where several look similar, and stores that one.

Then it either keeps the page or discards it. Discarding a crawled page is normal and it is not a penalty.

Why a crawled page gets left out

Because the engine judged it not worth storing. That usually means very little content, or content that repeats what another page already says.

Sets of near duplicate pages are the common case. Twenty city pages differing only in the place name look like one page to an indexer.

It can also be temporary. A new site with little authority sees pages crawled and held back until the site earns more trust.

The judgment is comparative rather than absolute. Your page is not being measured against a standard. It is being measured against everything else the engine already holds on that subject.

The judgment is comparative rather than absolute. Your page is measured against everything already held on the subject, not against a standard.

• Very little content on the page
• Content that repeats what another page already says
• A set of near-identical siblings where one is chosen
• A new site with little history, held back temporarily
• A page nothing links to, so nothing suggests it matters
• A page the engine judged simply not worth storing

Duplicates and canonicals

Where several addresses show the same content, the engine picks one to keep. Your canonical tag is a strong hint about which, not an instruction.

If the engine chooses a different one than you did, Search Console says so. That is worth checking when a page seems to have vanished.

Consistency helps. One address per page, one canonical, no mixture of trailing slashes and file extensions.

Where the engine chooses differently from your tag, it has decided two of your pages are the same thing, which is covered in canonical tags.

What you can control

Make each page genuinely different from its siblings, in what it says rather than in which word was swapped.

Give it a clear subject, a real answer near the top, and a title that matches what the page actually covers.

Link to it from somewhere relevant. An orphan page tells the engine nothing about how much you value it.

Depth helps when it is real. A page that answers the question, then answers the two questions that follow it, is doing something a thin page cannot. Padding to reach a word count is not the same thing and is easy to spot.

Instructions that keep a page out

A noindex instruction reliably keeps a page out of results. Blocking in robots does not, because a blocked page can still appear from links alone.

If you need a page out of search, let it be crawled and tell it to be excluded. The two instructions together cancel each other.

Check that no template is quietly applying noindex to pages you want found. It happens after most redesigns.

How ours are built for it

Every page written by Website Builder Studio carries a unique title, description and answer, and passes a check that runs before anything publishes.

Pages are measured against their siblings for how much they actually differ, and a page that fails that check does not publish.

That is the same judgment an indexer makes, applied before the page exists rather than discovered months later.

The measure is how much of a page survives once the words it shares with its siblings are stripped out. That is deliberately harsh, because it is roughly the test the engine applies when deciding whether it already has this page.

The measure is how much of a page survives once the words it shares with its siblings are stripped out — roughly the test an indexer applies.