Robots No Open Directory Project
Download List of All Websites using Robots No Open Directory Project
These are used if your website is listed in this directorie with information you do not want used in the results pages. This might be the case if you have old, outdated listings that no longer apply. They tell robots not to use information from these sources, and they are optional.
Vary Header check for User-Agent
Download List of All Websites using Vary Header check for User-Agent
When using the Vary: User-Agent header, caching servers should consider the user agent when deciding whether to serve the page from cache
Tags: