Google has announced that they are changing the way they crawl News content. Previously, Google used a robot which they called GoogleBot News to index news content, so if you wanted to protect that particular content, you needed to no-index for that Robot.
Starting now, the normal GoogleBot which indexes all the normal web content will also be crawling News content instead of the GoogleBot News.
This is how Google News started out – with the one Bot indexing everything – but then it changed to two, so that publishers could easily block certain content from either web or news.
Now they are going back to their original way – which is fine, because you can still do the either/or blocking, BUT, it does mean that some of the more detailed reporting in WebMasterTools will be lost.
What does this mean for news providers?
Your content can still be protected from being indexed by Google News by disallowing Googlebot-News in robots.txt (or using meta robots tag), Google will respect that robots.txt if it is more restrictive than the robots.txt entry for Googlebot.
For news sites which require login/payment/registration, Google will of course only be able to index that information which is available to the normal public.
This will not result in any changes for your news sitemap, which will be indexed as normal.
None, really. But this is a good time to remind news sites to;
- Implement a Google News Sitemap
- Remember that content hidden behind subscriber walls is of no value for SEO.