Telling search engines the moment something changes

The idea

Crawling is engines checking whether anything changed. Most of the time nothing has, which wastes their effort and yours.

This inverts it. Your site says a page changed, and the engines that care come and look.

It is a small protocol, openly published, and free to use.

It is worth understanding why this matters. Crawling at scale is expensive for everybody, and most of that expense is spent confirming that nothing changed. Announcing changes removes the guesswork from both sides.

Crawling at scale is mostly confirming that nothing changed, which is expensive for everybody. Announcing changes removes the guesswork from both sides.

Who participates

Bing and several other engines, including ones used more widely outside the United States.

Because a submission is shared among participants, telling one effectively tells all of them.

Google does not take part. That is the single most important limitation to understand.

The list has grown steadily since the protocol was published, and because submissions are shared between participants, telling one effectively tells all of them.

How it works in practice

You generate a key, host a file proving it is yours, and send the changed addresses when they change.

That is the whole mechanism. Most platforms that support it do all three steps without you thinking about it.

Submissions are usually reflected quickly, which is a real change from waiting on a crawl schedule.

The key file is the whole security model. Hosting a file with your key at the root of your domain proves you control the site, which is what stops anybody submitting addresses on your behalf.

The key file is the whole security model. Hosting a file with your key at the root proves you control the site, which stops anybody submitting addresses on your behalf.

What it does not do

It does not make a page rank. It does not force indexing. It does not affect Google at all.

It also does not replace a sitemap, which remains how engines discover the shape of your site.

Treat it as a speed improvement for discovery on part of the market, not as an optimisation.

It also does not replace good internal linking. Announcing a page tells an engine where to look. Linking it tells the engine how much the page matters, and only the second affects how it is treated.

It does not replace good internal linking. Announcing a page says where to look; linking it says how much the page matters — see internal linking.

When it matters most

Publishing often. A site adding pages weekly benefits more than one that changes twice a year.

Time sensitive content, where being found this week rather than next actually matters.

Large sites, where waiting for a crawler to work through everything takes a long time.

Removals are the underrated case. A page you took down can linger in results for weeks, and submitting it is the fastest way to have that checked.

It is also useful the day something goes wrong. A page published with an error, corrected within the hour, can be re-crawled promptly rather than sitting wrong for a fortnight.

• Publishing often, rather than a few times a year
• Time sensitive content
• Large sites, where waiting for a full crawl takes a long time
• Pages you have just removed and want dropped
• A price or availability change that matters today
• A fix you want re-crawled promptly

Removals too

You can submit an address that no longer exists, which tells participating engines to check and drop it.

That is faster than waiting for the engine to discover the page returns nothing.

It is a tidy way to handle a page you took down deliberately.

How it is used here

Publishing a site through Website Builder Studio submits the changed addresses as part of the release, so nothing waits on a crawler for the participating engines.

The sitemap is rebuilt from what actually exists at the same time, which is what the rest read.

Every page also passes a check that runs before anything publishes, so what is announced is a page that already met the standard.

Submission happens as part of publishing rather than as a separate chore, so it does not depend on anybody remembering after a busy release.

Everything announced has already passed the check that runs before anything publishes, so a submission is never asking an engine to look at something unfinished.