Comments on: Robots.txt Http and Https http://blog.super-networking.net/2008/03/robotstxt-http-and-https/ Tue, 07 Feb 2012 17:08:09 +0000 http://wordpress.org/?v=2.7.1 hourly 1 By: Super-Networking Blog » Blog Archive » Robots.txt Http and Https - Part II http://blog.super-networking.net/2008/03/robotstxt-http-and-https/comment-page-1/#comment-69856 Super-Networking Blog » Blog Archive » Robots.txt Http and Https - Part II Fri, 21 Mar 2008 19:18:09 +0000 http://blog.super-networking.net/systems/robotstxt-http-and-https/#comment-69856 [...] on splitting up the robots.txt you are showing based on if the bot was connecting on http or https here. I commented at the end that if this is an existing site and Google has already indexed your https [...] [...] on splitting up the robots.txt you are showing based on if the bot was connecting on http or https here. I commented at the end that if this is an existing site and Google has already indexed your https [...]

]]> By: chris.super http://blog.super-networking.net/2008/03/robotstxt-http-and-https/comment-page-1/#comment-68911 chris.super Mon, 17 Mar 2008 14:47:06 +0000 http://blog.super-networking.net/systems/robotstxt-http-and-https/#comment-68911 One thing to be aware of is this should be done when you are first setting up your site. You can run into problems if you put this into place on an existing site. What can happen is you tell Google not to reindex your https pages with your robots.txt but it will keep around the old ones it has already indexed. You cannot use the URL removal tool to get rid of your https version of your site unless you want to get rid of your http content as well. What you can do is add some meta tags to your https content to tell Google to get rid of the old content. I will try to post on how to do this at a later time. One thing to be aware of is this should be done when you are first setting up your site. You can run into problems if you put this into place on an existing site. What can happen is you tell Google not to reindex your https pages with your robots.txt but it will keep around the old ones it has already indexed.

You cannot use the URL removal tool to get rid of your https version of your site unless you want to get rid of your http content as well. What you can do is add some meta tags to your https content to tell Google to get rid of the old content. I will try to post on how to do this at a later time.

]]>