Skip to content
Snippets Groups Projects
Commit 4207218a authored by ale's avatar ale
Browse files

Merge branch 'noindex' into 'master'

Add a NGINX config option to set X-Robots-Tag:noindex on specific URLs

See merge request !265
parents 20b0bfc4 896c5691
No related branches found
No related tags found
1 merge request!265Add a NGINX config option to set X-Robots-Tag:noindex on specific URLs
# Global map used to forcefully prevent indexing of specific URLs.
# Configure via drop-in files in /etc/nginx/noindex/.
map $host$uri $noindex_header {
default '';
include /etc/nginx/noindex/*.conf;
}
URL deindexing
===
This list allows you to prevent bots from indexing specific URLs.
The string that is matched is the combination of the hostname and
the request path (without query arguments), i.e. for a request to
```
https://example.com/path/?arg=value
```
this map will attempt to match
```
example.com/path/
```
To deindex a URL, create or modify a file with a *.conf* extension in
this directory. The files should contain one entry per line, with the
[NGINX map module
syntax](http://nginx.org/en/docs/http/ngx_http_map_module.html):
```
"example.com/path/" noindex;
```
The value should always be set to `noindex`.
Deindexing via HTTP header is described here:
https://developers.google.com/search/docs/advanced/robots/robots_meta_tag
...@@ -75,3 +75,6 @@ location {{ ep.path }}/ { ...@@ -75,3 +75,6 @@ location {{ ep.path }}/ {
} }
{% endfor %} {% endfor %}
{% endfor %} {% endfor %}
add_header X-Robots-Tag $noindex_header;
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment