Robots No Open Directory Project
Download List of All Websites using Robots No Open Directory Project
These are used if your website is listed in this directorie with information you do not want used in the results pages. This might be the case if you have old, outdated listings that no longer apply. They tell robots not to use information from these sources, and they are optional.
Vary Header Accept-Encoding
Download List of All Websites using Vary Header Accept-Encoding
The Accept-Encoding request HTTP header advertises which content encoding, usually a compression algorithm, the client is able to understand. Using content negotiation, the server selects one of the proposals, uses it and informs the client of its choice with the Content-Encoding response header.
Tags: