Lionel Pairuna
←  back

one hard-coded literal  ·  ~128 pages

Site-wide noindex was impossible

The robots meta tag was hard-coded to all on every page. No code path could say noindex — not even for tag pages listing zero articles.

The SEO helper — every page, no exceptions 
function get_meta_data($record) {
    if (!$record) return false;

    return '
        <title>…</title>
        <meta name="robots" content="all">   ← fixed literal
    ';
}
No parameter, condition or override anywhere could change that value.

Context

After the previous deploy, Search Console still showed 200+ URLs under Crawled – currently not indexed. Most of them were news aggregation pages — categories, tags, author pages.

The problem

The SEO helper emitted <meta name="robots" content="all"> on every page of the site. There was no code path — no parameter, no condition, no override — that could produce noindex, not even for pages that clearly needed it: aggregation pages linking 0 or 1 article. A tag page with one article is redundant with that article; one with zero is an empty page. Google crawled them, declined to index them, and spent crawl budget doing it.

Evidence

Database measurement of the news section:

  • Tags: 87 of 116 (75%) linked 0 or 1 article
  • Authors / referrers: 41 of 51 (80%) linked 0 or 1 article
  • ~128 aggregation pages that added no value and could not be marked non-indexable

The fix

  1. Added a robots parameter to the SEO helper, default 'all' — no behaviour change unless a controller sets it explicitly.
  2. Added a subquery that counts the active records linked to each segment (count_records) — one aggregate, no N+1.
  3. In the news controller, derived the value:
$robots = ((int)($record['count_records'] ?? 0) <= 1)
    ? 'noindex, follow'
    : 'all';

<= 1, not == 0: a page listing a single article is redundant with that article. follow keeps the outbound links crawlable while the page itself stays out of the index.

Result

Baseline (Search Console, 2026-08-20, pre-deploy) — shared with the ghost-pages case, since both bugs fed the same buckets:

  • Crawled – currently not indexed: 245
  • Discovered – currently not indexed: 62

Verified now, in the production HTML:

  • a sample of thin pages emits <meta name="robots" content="noindex, follow">
  • a control page with several articles still emits content="all"
Expected after recrawl (~2–4 weeks): the ~128 thin pages move from Crawled – currently not indexed to Excluded by ‘noindex’ tag — a different bucket, the one for intentionally excluded pages. They don't disappear; they land in the correct category, and crawl budget shifts to real content. Before-and-after screenshots available on request.

What this shows

  • Reading a Search Console “Page indexing” report
  • Translating a coverage signal into a measurable, code-level content-quality rule
  • SQL: adding a linked-record count without an N+1
  • Verifying the rendered <meta robots> tag against production

Contact

Something similar in your Search Console? Tell me what's stuck — a screenshot or a single URL is enough to start.

lionel@lionelpairuna.dev