Technical SEO 2027: The Complete Plain-English Guide
pillar guide · crawling, indexing, speedHello my friend. Technical SEO is the work that lets a search engine reach, read, store and show your pages. This guide walks through every part of it in order, from robots.txt to Core Web Vitals, and each rule links to the Google page it comes from.
- every rule sourced
- audit checklist included
- last checked 11 Nov 2026
// example data. Google says not all pages make it through each stage. Technical SEO is the job of losing as few as possible on the way down.
Crawl, Render, Index: Where Technical SEO Lives
A page can only rank if Google can fetch it, render it and decide to store it. Technical SEO removes the blockers at each step.
// example files for illustration. A robots.txt block stops crawling, it does not remove a page from the index.
Head Tags Google Actually Reads
A small set of tags carries most of the technical signals. Here they are, with the checks I run.
<link rel="alternate" hreflang="de" href="https://example.com/de/page/">
<meta name="robots" content="index, follow">
x = ET.fromstring(requests.get('https://example.com/sitemap.xml').text)
urls = [e.text for e in x.iter() if e.tag.endswith('loc')]
print(len(urls), 'URLs in sitemap')
[...document.querySelectorAll('meta[name=robots]')].map(m => m.content)
// examples for illustration. A canonical is a strong hint, not a command.
What Technical SEO Is, and Why It Comes First
Technical SEO is everything you do to a website so search engines can crawl it, understand it and store it without trouble. It comes first because a page Google can't reach or index can't rank, however good the writing is.
Access
Can a crawler, the program that downloads your pages, actually get to them? This covers robots.txt, sitemaps, status codes, redirects and internal links.
Understanding
Once a page is fetched, can Google tell which version is the real one and what it's about? This covers canonical tags, noindex, JavaScript rendering, structured data and hreflang.
Experience
Does the page load fast, stay stable and work on a phone over a secure connection? This covers Core Web Vitals, mobile-first indexing and HTTPS.
Google's own starter guide describes SEO as helping search engines understand your content and helping people find your site. Technical SEO is the first half of that sentence. The second half belongs to content and on-page SEO, which shapes what each page says.
So think of it as plumbing. Nobody visits a house to admire the pipes, but everybody notices when the water stops. A site with clean technical work rarely gets praised for it. A site without it leaks traffic every single day.
What Technical SEO Is Not
It isn't a trick, and it isn't a score. A tool that shows "94 out of 100" tells you what that tool measured, not what Google thinks. Google states that it doesn't accept payment to crawl a site more often or rank it higher, so no plugin, badge or "submission service" can buy you a shortcut either. (Yes, people still sell those.)
What It Costs and How Long It Takes
If you do the work yourself, the main tools are free: Google Search Console, PageSpeed Insights and the Rich Results Test cost nothing. If I do it for you, my rate is €25 an hour, and technical work is part of every monthly plan further down this page.
As for time, be patient. Google's starter guide says some changes take effect in a few hours and others can take several months, and it suggests waiting a few weeks before you judge a change. A request for indexing in Search Console can take up to a week or two, and it isn't a guarantee. Anyone who promises faster is guessing.
How Google Crawls, Indexes and Serves Your Pages
Google Search works in three stages: crawling, indexing and serving results. Not every page gets through each stage, and Google says plainly that it doesn't guarantee to crawl, index or serve any page.
Discovery
links and sitemaps
Crawl
Googlebot fetches the URL
Render
JavaScript runs in Chrome
Index
content analyzed and stored
Canonical
one version picked
Serve
matched to a search
// the stages as described in Google Search Central's "In-depth guide to how Google Search works". The split into six boxes is mine, for clarity.
Stage One: Crawling
First, Google has to learn that a page exists. There's no central list of web pages, so Google keeps looking for new and updated URLs. It finds most of them by following a link from a page it already knows, and some through a sitemap you submit.
Then a program called Googlebot fetches the page. Googlebot decides by algorithm which sites to crawl, how often, and how many pages to fetch from each. It's also built to avoid crawling a site too fast, so it doesn't overload your server. If your server answers slowly or returns errors, crawling slows down.
During the crawl, Google renders the page. Rendering means it runs the page's JavaScript in a recent version of Chrome, much like your browser does, so that content added by scripts becomes visible to it.
Stage Two: Indexing
Next, Google tries to understand what it fetched. It analyzes the text and the key tags and attributes, such as the title element and image alt text. It also decides whether the page duplicates another page.
To do that, Google groups pages with similar content into a cluster and picks one as the canonical, which is the version that may be shown in results. The others in the cluster are alternates. Then the canonical page's information may be stored in the Google index, a very large database.
The word "may" matters. Google says indexing isn't guaranteed. Common reasons a page is left out are low content quality, a robots meta rule that disallows indexing, or a site design that makes indexing difficult.
Stage Three: Serving Results
Finally, when someone searches, Google looks through the index for matching pages and returns the ones it believes are the most relevant and of the highest quality. Relevance is determined by hundreds of factors, including the searcher's location, language and device. That's why "bike repair shop" shows different results in Paris than in Hong Kong.
Technical SEO mostly works on stages one and two. However, it touches stage three as well, because page experience signals such as Core Web Vitals are used by Google's ranking systems. More on those below.
Robots.txt and XML Sitemaps: the Doorman and the Map
A robots.txt file tells crawlers which URLs they may fetch, and an XML sitemap lists the URLs you want found. The first manages crawler traffic. The second helps discovery. Neither one controls indexing by itself.
Robots.txt Rules That Matter
The file must be named robots.txt and sit at the root of the host it applies to, for example https://www.example.com/robots.txt. A site can have only one, and it must be a UTF-8 text file. Its rules apply only to that host, protocol and port, so a subdomain needs its own file.
Google's crawlers support four fields: user-agent, allow, disallow and sitemap. Paths in rules are case-sensitive. The crawl-delay field isn't supported by Google, so adding it does nothing for Googlebot. Two wildcards work: the star matches any run of characters, and the dollar sign marks the end of a URL.
Two limits are worth knowing. Google reads only the first 500 kibibytes of the file, and it generally caches the file for up to 24 hours, so a change may not apply right away.
What Robots.txt Can't Do
Here's the part most people get wrong. Google says robots.txt is not a mechanism for keeping a web page out of Google. If other sites link to a blocked URL, that URL can still show up in results, just without a description. To keep a page out of the index, use noindex or put the page behind a password.
XML Sitemap Rules That Matter
A sitemap is a file that lists the pages, videos and other files on your site. One sitemap can hold up to 50,000 URLs or 50MB uncompressed. Beyond that you split it and, if you like, tie the parts together with a sitemap index file. The file must be UTF-8 encoded and use full, absolute URLs.
Google ignores the priority and changefreq values, so don't waste an afternoon tuning them. It does use lastmod, but only when the date is consistently accurate and reflects a significant change to the page.
Do you even need one? Google says you might not if your site is small, which it defines as about 500 pages or fewer, and well linked internally. You probably do if the site is large, new with few external links, or heavy on images and video. A sitemap costs nothing, so I add one anyway. Just remember that listing a URL doesn't guarantee it gets crawled or indexed.
You can submit the sitemap in Search Console's Sitemaps report, or add a Sitemap line to robots.txt. If you want a second pair of eyes on yours, my SEO audit service checks both files first.
Disallow: /cart/
Disallow: /checkout/
Disallow: /*?sort=
Allow: /
Sitemap: https://www.example.com/sitemap.xml
// example file. It keeps crawlers out of the cart, the checkout and sorted duplicates, and points to the sitemap. Blocking is not the same as removing from the index.
| rule | value |
|---|---|
| max URLs per sitemap | 50,000 |
| max size, uncompressed | 50MB |
| priority, changefreq | ignored |
| lastmod | used if accurate |
| "small site" in Google's words | about 500 pages or fewer |
// source: Google Search Central, "Build and submit a sitemap" and "Learn about sitemaps".
Canonical Tags, Noindex and Duplicate URLs
A canonical tag tells Google which of several similar URLs is the main one, and noindex tells Google to keep a page out of the index entirely. They solve different problems, so use the one that matches the job.
| What it does | Use it when | |
|---|---|---|
| robots.txt disallow | Stops crawling of the URL | Crawlers waste time on useless URLs, such as sorted or filtered lists |
| noindex | Keeps a crawlable page out of the index | A page must exist for visitors but shouldn't appear in search |
| rel="canonical" | Names the preferred URL among duplicates | The same content lives on several URLs and all must stay online |
| 301 redirect | Sends visitors and crawlers to a new URL | The old URL is retired for good |
| password | Blocks everyone without a login | The content is private |
// summary of Google Search Central guidance on robots.txt, noindex, canonicalization and redirects. The wording of the table is mine.
Duplicate Content Is Not a Penalty
Let's clear up an old scare first. Google's starter guide says duplicate content on your own site is not a violation of its spam policies. It calls it inefficient, not something that causes a manual action. The real cost is wasted crawling and Google picking a URL you didn't want.
Duplicates usually come from boring places: http and https versions, www and non-www, trailing slashes, tracking parameters, and product lists that can be sorted six ways.
How to Tell Google Your Preferred URL
Google lists the methods by strength. A redirect is a strong signal that the target should become canonical. A rel="canonical" link is also a strong signal. Including the URL in a sitemap is a weak signal. You can combine them, and combining them raises the odds that your preferred URL wins.
A canonical link sits in the head of the page and looks like this: <link rel="canonical" href="https://www.example.com/shoes/">. It works for HTML pages only. For files such as PDFs, the same instruction can be sent as an HTTP header.
Then back the signal up. Link internally to the canonical URL, not to a duplicate. Prefer the HTTPS version, because Google prefers HTTPS pages over equivalent HTTP pages as canonical. And don't use robots.txt or noindex to choose a canonical. Google warns against both, because a blocked page can't be read and a noindex page is removed from Search completely.
How Noindex Works
Noindex is a rule you set in one of two ways. The first is a meta tag in the page's head: <meta name="robots" content="noindex">. The second is an HTTP response header, X-Robots-Tag: noindex, which also works for non-HTML files.
One condition is essential. For noindex to work, the page must not be blocked by robots.txt. If the crawler can't fetch the page, it never sees the rule, and the URL can stay in the index. Blocking a page in robots.txt and adding noindex to it is the most common contradiction I find in audits.
If rules conflict, for example one tag says index and another says noindex, Google applies the more restrictive one. Both rules matter just as much when a whole site moves, which is why migrations get their own checklist.
HTTP Status Codes and Redirects
Every time a crawler asks for a URL, your server answers with a three-digit status code, and Google treats each group of codes differently. Get the code right and Google does the sensible thing. Get it wrong and good pages drop out or dead pages hang around.
How Google Reads Each Group
A 200 means success, so the content is passed on for indexing. If a 200 page shows an error message or is empty, Search Console reports it as a soft 404, which means "this says OK but looks like an error".
A 3xx code is a redirect. Google treats a 301 as a strong signal that the target should be processed, and a 302 as a weak one. Codes 308 and 307 behave like 301 and 302. By default, Google's crawlers follow up to 10 redirect hops and then give up.
A 4xx code is a client error. Google doesn't use content from URLs that return 4xx, and already indexed URLs that start returning 4xx are removed from the index. The one exception is 429, "too many requests", which Google treats as a server error.
A 5xx code is a server error. These, along with 429, make Google's crawlers slow down for a while. Indexed URLs are kept at first, but they're eventually dropped if the errors continue. Once the server answers with 2xx again, the crawl rate gradually recovers.
Redirects Done Properly
Use a permanent, server-side redirect whenever you can, because Google says that type is interpreted most reliably. Use it only when the move is final. With a permanent redirect the target is treated as canonical. With a temporary one it isn't, so the old URL may stay in results.
JavaScript redirects are a last resort, for when you can't set a server-side or meta refresh redirect. In addition, avoid chains. Google advises redirecting straight to the final destination, and keeping redirects for as long as possible, generally at least one year after a move.
Worried about losing value through a redirect? Google states that 301 and other permanent redirects don't cause a loss in PageRank. So, redirect retired pages to their closest live equivalent and move on. If a whole site is moving, read my notes on the website migration SEO service before you touch anything.
Two Habits to Drop
First, don't answer missing pages with a 200 and a "sorry, not found" message. Return a real 404, or a 410, so Google knows the page is gone. Second, don't use 401 or 403 to slow Googlebot down. Google says those codes aren't for limiting crawl rate.
| code | meaning | Google's reaction |
|---|---|---|
| 200 | OK | index content considered |
| 301 / 308 | moved for good | strong target becomes canonical |
| 302 / 307 | moved for now | weak old URL may stay |
| 404 / 410 | gone | drop removed from index |
| 429 | too many requests | slow treated as server error |
| 500 / 503 | server error | slow dropped if it persists |
| 200 + error text | soft 404 | flag reported in Search Console |
// source: Google Search Central, "How HTTP status codes, and network and DNS errors affect Google Search". Shortened by me.
Site Architecture, Internal Links and Crawl Budget
Site architecture is the way your pages are grouped and linked together, and it decides how easily crawlers and people find each page. For most sites, good internal linking matters far more than crawl budget, which only becomes a real concern on very large or very fast-changing sites.
Links Google Can Follow
Google can generally crawl a link only if it's an a element with an href attribute. A button that changes the page through JavaScript, or a span dressed up as a link, may not be followed. So check your menus, filters and "load more" buttons first.
Google also says every page you care about should have a link from at least one other page on your site. A page with no internal links is called an orphan, and orphans are found late or never. Anchor text, the clickable words of a link, should be descriptive, reasonably concise and relevant. "Click here" tells nobody anything. Internal links also feed on-page SEO, because the anchor text tells Google what the target page is about.
URLs and Folders
Use readable words in URLs, not long ID numbers, and separate words with hyphens instead of underscores. Remember that Google's URL handling is case-sensitive, so /Shoes/ and /shoes/ are two addresses. Avoid session IDs in URLs where possible, and don't use the part after a # to load different content, because Google generally doesn't support fragments for that.
Grouping similar pages in folders also helps. According to the starter guide, directories help Google learn how often the URLs inside them change. A news folder gets visited more often than a terms-and-conditions folder, as it should.
Crawl Budget: Who Needs to Care
Crawl budget is the set of URLs Google can and wants to crawl on your site. It's shaped by two things: the crawl capacity limit, meaning how much your server can take, and crawl demand, meaning how much Google wants your content.
Google's crawl budget guide is written for three kinds of sites, listed in the panel. It calls the numbers rough estimates, not exact thresholds. If you run a 60-page site for a bakery in Leeds, this isn't your problem. Google's advice for everyone else is simple: keep the sitemap up to date and check your index coverage regularly.
If you do qualify, the fixes are practical. Consolidate duplicate content. Block useless URLs, such as endless filter combinations, with robots.txt. Return 404 or 410 for pages that are gone for good. Remove soft 404s, keep sitemaps current, avoid long redirect chains and make pages load faster. Online stores with filters are the usual candidates, and my ecommerce SEO services start there.
Log Files in Brief
A server log is a file where your server records every request it receives, including each visit from Googlebot. Reading it shows what was really crawled, not what you hoped was crawled. Because anyone can pretend to be Googlebot, Google explains how to verify a visitor: run a reverse DNS lookup on the IP address and check that the name ends in googlebot.com, google.com or googleusercontent.com.
No log access? Search Console has a Crawl Stats report under Settings. Google aims it at advanced users and says sites with fewer than a thousand pages shouldn't need it. I agree. Small sites have better things to do.
├─ /cakes/
│ ├─ /cakes/birthday/
│ └─ /cakes/wedding/
├─ /bread/
│ └─ /bread/sourdough/
├─ /order/
└─ /blog/
└─ /blog/how-to-store-sourdough/
// example data. Every page sits in a topic folder and is reachable through plain links from the home page.
// source: Google's "Optimize your crawl budget" guide, which also includes sites with a large share of URLs marked "Discovered - currently not indexed". Bar lengths are illustrative, not to scale.
JavaScript Rendering, Mobile-First Indexing and HTTPS
Google can run JavaScript, it indexes the mobile version of your site, and it prefers secure pages. Each of those facts carries a few rules, and breaking them hides content from Google without any error message.
// source: Google Search Central, "Mobile site and mobile-first indexing best practices".
JavaScript Rendering Basics
Google processes JavaScript pages in three phases: crawling, rendering and indexing. After the first fetch, the page waits in a render queue. Google says that wait may be a few seconds, but it can take longer. So content that only appears after scripts run is indexed later than content in the plain HTML.
Because of that, Google still calls server-side rendering or pre-rendering a great idea. Both mean the server sends finished HTML. It's faster for visitors, and not every bot can run JavaScript.
A few rules save most JavaScript sites:
- Use real links: a elements with an href attribute.
- For page-to-page navigation inside an app, use the History API, not URL fragments.
- Return meaningful status codes. If a single-page app can't, redirect missing pages to a URL that returns 404, or add noindex with JavaScript.
- Never put noindex in the original HTML and hope to remove it with a script. When Google sees noindex, it may skip rendering altogether.
- Don't change the canonical URL with JavaScript to something different from the one in the original HTML.
To see what Google actually rendered, use the URL Inspection tool in Search Console and open the rendered HTML. If your main text isn't in there, Google doesn't have it.
Mobile-First Indexing
Mobile-first indexing means Google uses the mobile version of your content, crawled with its smartphone agent, for indexing and ranking. In other words, what your site shows on a phone is your site as far as Google is concerned.
Google recommends responsive design, where one URL serves the same HTML and the layout adapts to the screen. It's the easiest pattern to build and maintain. If you have a separate mobile version, the mobile and desktop pages need the same content, the same robots meta tags, the same structured data, equivalent titles and meta descriptions, and the same image alt text.
One more trap: don't lazy-load primary content upon user interaction. Googlebot doesn't swipe, click or type, so content that appears only after a tap stays invisible.
HTTPS
HTTPS is the secure version of the web protocol, shown by the padlock in the browser. Google's page experience guidance asks, as one of its self-check questions, whether your pages are served in a secure fashion. Google also prefers HTTPS pages over equivalent HTTP pages as canonical, unless something is wrong, such as an invalid certificate.
So the setup is short. Install a valid certificate. Redirect every HTTP URL to its HTTPS twin with a permanent redirect. Update internal links, canonical tags and the sitemap to the HTTPS addresses. Google counts a move from HTTP to HTTPS as a site move with URL changes, which is why the redirects matter.
Core Web Vitals: the Three Numbers to Hit
Core Web Vitals are three measurements of real visitor experience: how fast the main content loads, how quickly the page reacts, and how much the layout jumps. A page passes when it hits the "good" mark for each one at the 75th percentile of page loads.
The Thresholds
Largest Contentful Paint, or LCP, measures loading. It should happen within 2.5 seconds of the page starting to load. Interaction to Next Paint, or INP, measures responsiveness, and a good score is 200 milliseconds or less. Cumulative Layout Shift, or CLS, measures visual stability, and a good score is 0.1 or less.
The "75th percentile" part means that at least 75 out of 100 real page loads must meet the mark, counted separately for mobile and desktop. So one fast test on your office Wi-Fi proves very little. The numbers come from real Chrome users, collected in the Chrome User Experience Report, which is called field data.
Do They Affect Rankings?
Yes, modestly. Google says Core Web Vitals are used by its ranking systems, and it highly recommends achieving good scores. However, it also says there is no single page experience signal, and that Search tries to show the most relevant content even when the page experience is sub-par.
So a fast page with nothing useful on it won't outrank a slower page that answers the question. Speed is a tiebreaker and a courtesy to your visitors. It is not a magic lever, whatever the plugin's sales page says.
How to Measure and Fix Them
Start with PageSpeed Insights, which shows field data for a URL when enough exists, plus a lab test. Then open the Core Web Vitals report in Search Console, which groups similar URLs and labels each group Good, Need improvement or Poor. Lighthouse and Chrome DevTools help you debug, but Lighthouse is a lab tool and can't measure INP. It uses Total Blocking Time as a stand-in.
The usual fixes, in my experience, are unglamorous. For LCP, shrink and compress the main image and cut slow server responses. For INP, remove heavy scripts you don't need. For CLS, give images and ad slots fixed dimensions so nothing jumps. WordPress owners can get the same fixes done through my WordPress SEO services.
| metric | good | poor |
|---|---|---|
| LCP (loading) | 2.5 s or less | over 4 s |
| INP (response) | 200 ms or less | over 500 ms |
| CLS (stability) | 0.1 or less | over 0.25 |
// source: web.dev, "Web Vitals" and "How the Core Web Vitals metrics thresholds were defined". Anything between good and poor is "needs improvement".
// example data: a taxi firm in Sydney. It shows how a before and after report reads, not a client result.
Structured Data and Hreflang in Brief
Structured data is a standard format for describing a page to search engines, and hreflang tells Google which language or country version of a page to show. Both are optional. Both help only when they are accurate.
Structured Data Basics
Structured data is extra code that labels what's on the page: this is a product, this is its price, this is the author. Google supports three formats, JSON-LD, Microdata and RDFa, and generally recommends JSON-LD if your setup allows it.
The reward can be a rich result, which is a search listing with extras such as review stars or a product price. However, Google doesn't guarantee that structured data will show up in results, even when the markup is correct. The rules are strict, too:
- Mark up only content that visitors can see on the page.
- Don't mark up misleading content, such as fake reviews.
- Include every required property, or the item isn't eligible.
- A few complete, accurate properties beat many sloppy ones.
Breaking the rules can bring a manual action. According to Google, that removes the page's eligibility for rich results, though it doesn't affect how the page ranks in web search.
Also, check that a feature still exists before you build it. For example, Google's FAQ documentation states that the FAQ rich result is no longer shown in Search results as of June 2026. FAQ markup is still valid Schema.org code, but it won't earn that display. Test your code with the Rich Results Test before you publish it.
Hreflang in Brief
If the same page exists for several languages or regions, hreflang annotations tell Google about the variations. There are three ways to add them, and Google treats them as equal: link tags in the HTML head, HTTP headers, or the sitemap.
The rules are short, and they're where everyone slips. Each version must list itself as well as all other versions. The links must point both ways, because if two pages don't point to each other, the tags are ignored. URLs must be fully qualified, including https. Language codes follow ISO 639-1, and optional region codes follow ISO 3166-1 Alpha 2. So "en-GB" is valid and "en-UK" is not, because UK isn't the ISO code for the United Kingdom.
Finally, the reserved value x-default marks the fallback page for visitors who match no listed language. And no, hreflang doesn't tell Google what language a page is in. Google says it doesn't use hreflang or the HTML lang attribute to detect language. It works that out from the page itself.
"@context": "https://schema.org",
"@type": "Article",
"headline": "How to Store Sourdough",
"datePublished": "2026-11-11",
"author": { "@type": "Person", "name": "the name shown on the page" }
}
// example markup. Google lists no required properties for Article, only recommended ones such as headline, author and dates.
<link rel="alternate" hreflang="en-gb" href="https://example.com/uk/">
<link rel="alternate" hreflang="x-default" href="https://example.com/">
// example tags. The same three lines go on all three pages, so every version lists itself and the others.
Technical SEO Audit Checklist, Step by Step
A technical SEO audit is a fixed list of checks, run in order, from "can Google reach the site at all" to "is the site pleasant to use". Run it in this order, because a problem early in the list can hide everything after it.
| # | check | where to look | a pass looks like |
|---|---|---|---|
| 1 | Site is reachable | open the home page, then Crawl Stats | 200 responses, no 5xx spikes |
| 2 | robots.txt is sane | yoursite.com/robots.txt | blocks only cart, search and filter URLs |
| 3 | Sitemap is submitted | Search Console, Sitemaps report | status Success, URLs match real pages |
| 4 | Key pages are indexed | Page indexing report, URL Inspection | money pages show as indexed |
| 5 | No stray noindex | URL Inspection on key pages | no noindex rule on pages you want found |
| 6 | One version of each URL | canonical tags, redirects | http, www and slash variants all redirect |
| 7 | Redirects are clean | a crawler, or a manual check | one hop, permanent, to a live page |
| 8 | Internal links work | a crawler | no orphans, no links to 404 pages |
| 9 | Mobile page matches desktop | URL Inspection, rendered HTML | same main content, tags and data |
| 10 | Core Web Vitals are good | PageSpeed Insights, Search Console | LCP 2.5 s, INP 200 ms, CLS 0.1 or better |
// the order and wording are mine, built from the Google Search Central and Search Console Help pages listed in the sources. Tick each row, then move on.
Why the Order Matters
If robots.txt blocks the whole site, nothing after check two can be fixed by editing pages. So fix access first, then understanding, then experience. That is also the order of the three parts at the top of this guide.
Which Tools You Need
Google Search Console is free and tells you what Google itself saw. A crawler, which is a program that visits your pages the way Googlebot does, shows the problems Google hasn't reported yet. PageSpeed Insights covers speed. For a small site, those three are enough. If you'd rather hand the whole list to someone, my SEO audit service runs it for €25 an hour and returns one page of fixes sorted by impact.
How to Read Search Console's Indexing Statuses
The Page indexing report tells you, URL by URL, why Google did or didn't index a page. Several of its labels sound scary and aren't, so read the definition before you panic.
| status | Google's meaning | usual action |
|---|---|---|
| Crawled - currently not indexed | The page was crawled by Google but not indexed. | review often needs nothing |
| Discovered - currently not indexed | The page was found by Google, but not crawled yet. | wait then check server speed |
| Blocked by robots.txt | This page was blocked by your site's robots.txt file. | check is the block intended? |
| Excluded by noindex | Google found a noindex directive, so it did not index the page. | check is the rule intended? |
| Duplicate without user-selected canonical | A duplicate of another page, with no preferred canonical stated. | set a canonical or redirect |
| Soft 404 | A friendly "not found" message without a 404 response code. | fix return a real 404 |
| Page with redirect | A non-canonical URL that redirects to another page. | fine the target may be indexed |
| Alternate page with proper canonical | An alternate that correctly points to the indexed canonical. | fine no action |
// source: Search Console Help, "Page indexing report". Definitions shortened by me; the action column is my opinion.
Two details save a lot of worry. First, the report has a Source column showing whether the cause is Google or your website, and you can generally fix only the issues whose source is Website. Second, "Crawled - currently not indexed" is common, and Google says it doesn't necessarily need attention, because Google doesn't index every page on the web.
For example, a plumber in Austin with 30 service pages might see two thin tag pages listed as crawled but not indexed. That's a shrug. The same status on the "emergency plumber" page is a problem worth a closer look at the content and internal links.
Six Technical SEO Mistakes and Their Fixes
Most technical problems I meet are old, boring and easy to fix once someone looks. These six come up again and again, and each fix is a single change.
Blocking and noindexing the same page
Google can't read a noindex rule on a page that robots.txt blocks. Fix: remove the robots.txt block, keep the noindex, and let Google fetch the page.
Leaving noindex on after launch
Many site builders let you hide a staging site from search, and the setting is easy to forget at launch. Fix: inspect your key pages in URL Inspection the day you go live.
Soft 404 pages
A "not found" message served with a 200 code confuses Google. Fix: return a real 404 or 410, or redirect to the closest live page.
Redirect chains
Old redirects stack up after several redesigns. Fix: point every old URL straight at its final destination, in one hop.
Links that aren't links
Menus built from buttons or spans may not be followed. Fix: use a real a element with an href attribute for every navigation item.
A thinner mobile page
Google indexes the mobile version, so content hidden there is gone. Fix: make mobile and desktop carry the same main content, tags and structured data.
// the fixes follow Google Search Central guidance on noindex, status codes, redirects, crawlable links and mobile-first indexing. The ordering and the launch-day habit in mistake 2 are my own experience, offered as opinion.
How to Check That a Fix Worked
You verify a technical fix by testing the live URL, asking Google to recheck, and then waiting. Each of those steps has its own limits, so here are the numbers.
Test the live URL
In URL Inspection, run a live test. It fetches the URL in real time, so it shows your fix even before Google recrawls. A pass isn't a promise that the page will appear in search.
Ask Google to recheck
For one page, use Request indexing. Google says it doesn't guarantee the page will appear, and there is a daily limit. For many pages, submit a sitemap, or use Validate fix on the issue in the Page indexing report.
Wait, then compare
Google says validation typically takes up to about two weeks, and sometimes much longer. Speed fixes are tracked over a 28-day window in the Core Web Vitals report.
// source: Search Console Help pages "URL Inspection tool", "Page indexing report" and "Core Web Vitals report".
Two Rules for Validation
Fix every instance of an issue before you click Validate fix, because Google says validation stops if even one instance remains. After you click it, don't click it again until validation has succeeded or failed.
What the Rendered HTML Tells You
URL Inspection can show the raw HTML and a screenshot of the rendered page, but the screenshot is available only in a live test. If your main text is missing from the rendered HTML, fix that before anything else, because Google can't index what it can't see.
How I Do Technical SEO for You
I run the checklist above on your site, fix what I can fix myself, and write down the rest in plain English. You see each change and why I made it. It's the same ten checks, every time, in the same order.
Because I'm also a WordPress developer, I can usually make the change instead of handing you a to-do list. For other platforms I tell your developer exactly what to do. No guarantees on rankings or traffic, from me or from anyone honest. What I can promise is that each fix is checked and documented.
what_i_do
- Full crawl and Search Console review
- robots.txt, sitemap and canonical fixes
- Redirect and status code cleanup
- Speed work on the slowest templates
what_you_get
- A fix list sorted by impact
- The fixes made, not just listed
- Before and after numbers per fix
// example data: a 30-page dental site in Toronto. It shows the report format, not a client result.
Technical SEO Inside Three Plans, One Hourly Rate
Technical work is part of every plan, next to on-page, local and off-page link building. A bigger plan buys more work each month. A one-off technical job is €25 an hour.
Starter
For a small site that needs steady, careful work.
€475/month
- On-page, technical, local and off-page
- 1 to 2 guest post links a month
- Links from blogs with real traffic
- Monthly plain-English report
Growth
For a site with more pages and more fixes waiting.
€875/month
- On-page, technical, local and off-page
- 2 to 3 guest post links a month
- Links from blogs with real traffic
- More pages and fixes handled each month
Full Campaign
For a big clean-up plus steady link building.
€1,275/month
- On-page, technical, local and off-page
- 5 or more guest post links a month
- Links from blogs with real traffic
- The most work per month of the three
Set Your Own Budget
If none of the three plans fit, tell me what you can spend each month and I will tell you honestly what I can do for it. A small budget goes to the most urgent technical fixes first.
// prices and plan contents are my own. Technical work, links and on-page work share the same monthly hours. If you also need local or off-page help, see my guides to local SEO and off-page SEO. Or look at my cheap SEO services as a whole.
Technical SEO Questions and Answers
Short answers first, with the detail above.
What is technical SEO in simple words?
Technical SEO is the work that makes sure search engines can reach, read and store your pages. It covers robots.txt, sitemaps, redirects, duplicate URLs, mobile pages, HTTPS and page speed.
Is technical SEO more important than content?
Neither wins, because they do different jobs. A page Google can't index can't rank, however good it is, and an indexed page with nothing useful on it won't rank well either. Fix access first, then the writing.
How long does technical SEO take to work?
Google says changes can take effect in a few hours or take several months, and it suggests waiting a few weeks before judging them. Search Console validation typically takes up to about two weeks.
Do Core Web Vitals affect rankings?
Yes, modestly. Google says its ranking systems use Core Web Vitals and recommends good scores, but it also says relevance still comes first. A fast, empty page won't beat a slower page that answers the question.
Does robots.txt remove a page from Google?
No. Robots.txt controls crawling, not indexing, and a blocked URL can still appear in results without a description. To keep a page out of Google, use noindex or a password.
Is duplicate content a penalty?
No. Google says duplicate content on your own site isn't a violation of its spam policies. The cost is wasted crawling and Google choosing a URL you didn't want, which a canonical tag or redirect fixes.
Do I need an XML sitemap?
Not always, but it's free and rarely hurts. Google says a small site of about 500 pages or fewer that is well linked may not need one, while large, new or media-heavy sites probably do.
Can I do technical SEO myself?
Yes, for a small site. Search Console, PageSpeed Insights and the Rich Results Test are free, and the checklist above covers the basics. Hire help when a fix touches code, hosting or a site move.
How much does technical SEO cost with you?
My rate is €25 an hour for a one-off job. Technical work is also part of every monthly plan: Starter €475, Growth €875 and Full Campaign €1,275, or a monthly budget you set yourself.
Send Me Your URL. I'll Check the Technical Basics.
Message me on WhatsApp with your website address. I'll reply with an honest first look at what Google can and can't reach on your site, and what I'd fix first.
// prefer email? [email protected] works too. Take care, my friend.
Sources and Last Check
- Google Search Central: In-depth guide to how Google Search works (crawling, indexing, serving)
- Google Search Central: SEO Starter Guide (timing, duplicate content, URL habits)
- Google Search Central: Introduction to robots.txt (what it can and cannot do)
- Google Search Central: Learn about sitemaps (who needs one)
- Google Search Central: Build and submit a sitemap (limits, ignored fields)
- Google Search Central: Consolidate duplicate URLs (canonical signals)
- Google Search Central: Block search indexing with noindex (meta tag and header)
- Google Search Central: HTTP status codes, and network and DNS errors (how each code is treated)
- Google Search Central: Redirects and Google Search (permanent and temporary redirects)
- Google Search Central: Make your links crawlable (a elements, anchor text)
- Google Search Central: Optimize your crawl budget (who it is for)
- Google Search Central: JavaScript SEO basics (rendering, links, status codes)
- Google Search Central: Mobile-first indexing best practices (content parity)
- Google Search Central: Core Web Vitals (ranking use, tools)
- web.dev: Web Vitals (thresholds at the 75th percentile)
- Google Search Central: Intro to structured data (formats, guidelines)
- Google Search Central: FAQ structured data (FAQ rich result removal notice, June 2026)
- Google Search Central: Localized versions and hreflang (annotation rules)
- Search Console Help: Page indexing report (status definitions, validation)
- Search Console Help: URL Inspection tool (live test, request indexing)
- Search Console Help: Core Web Vitals report (grouping, 28-day tracking)
// last checked 11 Nov 2026. Prices and plan contents are my own. Opinions and example data are marked where they appear.