include email support in robots.txt

[DmC]

I'm referring to the https://helionet.org/index/forum/91-email-support sub forum.

Email addresses etc etc are usually appearing there especially on replies.

And people are not always aware of it.

Excluding it is bullet proof and obviously somebody can visit the forum and see the threads but at least Google and the other reputable engines won't include nor return results.

PS: It would be even better to make it visible to registered users only.

 

Edited December 20, 2024 by DmC

[wolstech]

It’s already set to be invisible to guests…only logged in users can see that forum.

[DmC]

1 hour ago, wolstech said:

It’s already set to be invisible to guests…only logged in users can see that forum.

Maybe include it in the robots file too, it's pretty pretty simple and won't harm 😊

[wolstech]

Invision manages its own robots.txt, and there doesn't appear to be an option to exclude a single forum without using the custom config option instead of the IPB recommended. I'd do that, but I'm not sure if custom rules replace the default ones or just be added to them. That said, Google can't index email support anyway because it cannot sign in, and just gets an access denied error if it tries to go in there. Preventing search indexing of that forum is the main reason it's hidden.

Also, even if I set up custom robots rules in IPB, I'm not even sure the robots.txt even does anything in our environment since IPB puts it in /index/ instead of in the domain root: https://helionet.org/index/robots.txt