Super-Networking Blog

Holy cow! Can you believe that finally the developers of Wireshark have reached 1.0.0 of their product. I have been running 0.9X versions of this software for what seems like forever.

Wireshark for you who haven’t looked for a long time used to be the packet sniffer called Ethereal. This is a great free packet sniffer that every Network Admin worth his salt should have on his laptop and or Thumb Drive.

Download it here

Older Post on Wireshark/Ethereal:

Wireshark aka Ethereal

Ran across something interesting this week. I was looking into performance on some of our webservers and noticed that there would be unexplained CPU spikes every once in a while. No real spikes in traffic, no additional tasks going on.

In the event logs I did find something that correlated with the CPU spikes:

Event ID: 7036

“The WinHTTP Web Proxy Auto-Discovery Service service entered the running state.”

Event ID: 7036

“The WinHTTP Web Proxy Auto-Discovery Service service entered the stopped state.”

This happens many times a day, starting and stopping. When in transition it seemed to spike the CPU for a little while.

Service Description is the following:

Implements the Web Proxy Auto-Discovery (WPAD) protocol for Windows HTTP Services (WinHTTP). WPAD is a protocol to enable an HTTP client to automatically discover a proxy configuration. If this service is stopped or disabled, the WPAD protocol will be executed within the HTTP client’s process instead of an external service process; there would be no loss of functionality as a result.

In looking into this more I found this is not needed to run IIS or anything else on your server. Only needed for your browser on your server to detect a proxy for Internet browsing. So do yourself a favor and disable this service, you will see it most likely set as manual and stopped but don’t let this fool you. Check out your event logs and if you see it starting and stopping disable it.

So if you are running Windows 2003 with IIS 6 and all of your web applications are running ASP .Net 2.0 you might want to get rid of .Net 1.1.

Well with Windows 2003 it isn’t easy or advisable to completely remove .Net 1.1 but you can unregister is from IIS having the same affect as far as your websites are concerned.

You need to open up the command prompt and go to the path .Net is installed in. Typically is C:\Windows\Microsoft.NET\Framework\v1.1.4322\

If you have other versions of .net installed you will see all the versions under Framework as well, be sure to go into the 1.1 directory.

From the command prompt in that directory type the command “aspnet_regiis.exe -u”

This will unregister this version of the framework with IIS and in turn IIS will not see it. You will want to do a reboot to make sure everything comes up clean afterwards.

Also make sure that you do not have any websites using .Net 1.1 because you will most likely mess it up when it is forced over to .Net 2.0

So I posted earlier on splitting up the robots.txt you are showing based on if the bot was connecting on http or https here. I commented at the end that if this is an existing site and Google has already indexed your https content you need to be careful. Once you block Google from reindexing your https content using the robots.txt file Google will ignore changes, but it will keep the old indexed pages.

You cannot use the URL removal tool because you can’t tell it to only remove https urls. The way you get around this is to remove the robots.txt restrictions from https and put meta tags for Google to remove the content only on our https content.

Tags you are going to want to user are:

<meta name=”ROBOTS” CONTENT=”NONE”>

and

<meta name=”GOOGLEBOT” content=”NOARCHIVE”>

This will tell Google to not index the pages and remove any old caches and indexes. This could take some time for Google to crawl all of your https content. Once everything you want out of their index is gone you can go ahead and change over to the easier https robots.txt file restrictions.

I do not have the code that allows you to set different meta tag info based on http or https. If someone has that please comment on this post. This should help out the SEO on your site. Thanks

Resource Page for You

Do you need something to monitor the bandwidth usage on one server? For instance if you have a hosted server that has limited total usage. Well if you want a simple free tool download NetMeter.

You can get real time statistics, totals including day, week and month. It will also project your usage over the day, week or month based off what you have already used. It is easy to use and best of all FREE!

One of the big things a lot of people are doing these days is SEO or search engine optimization. I am not planning on going over a comprehensive guide for SEO right now but as I run into them in my own job I will post them here.

One of the more recent tasks I had was setting up a different robots.txt for our http site vs our https site. It is the same content but Google indexes them as two different copies of the content. Google then penalizes your site for duplicate content. Well you setup your robots.txt file to exclude the files and directories you want from the bots but it doesn’t help the http vs http. Both protocols will use the same robots.txt file and so you will end up with duplicate content in Google’s index.

Here is how to solve this problem when you are running IIS 6.0 on a Windows 2003 server. You will need to also be running ASP .Net 2.0 for this solution to work. You might be able to get this to work on other platforms but I have not tested them. What we will be doing here is creating a dynamic robots.txt file, it is only one file but it will display different results depending on if you connect with http or https.

1) Create your robots.txt file:

<%@ WebHandler Language="C#" Class="MyNamespace.robotshandler" %>

using System;
using System.Web;

namespace MyNamespace {

    public class robotshandler: IHttpHandler {

        public void ProcessRequest (HttpContext context) {

            context.Response.ContentType = "text/plain";
            context.Response.Write("User-agent: *\n");

            if (context.Request.ServerVariables["Https"]=="off"){
                // HTTP
                context.Response.Write("Allow: /\n");
                context.Response.Write("Disallow: /MyDisallowedDirectory/\n");
            } else {
                // HTTPS
                context.Response.Write("Disallow: /");
            }
        }

        public bool IsReusable {
            get {return false;}
        }
    }
}

2) IIS needs to have .txt files passed through ASP .Net

- Open IIS and right click on your website and bring up the properties screen

- Go to Home Directory > Configuration. You will be on the Mappings Tab.

- Locate the ASPX item and click Edit - Copy the path in the Executable Field and cancel out of that window.

-Cancel

-Click “Add”

-Populate the Executable path with the value you copied in the last section

-Extension “.txt”

-Enter “GET” in the “Limit To” field

-Save and Exit

3) Need to modify web.config to process the .txt correctly

-Add the following under system.web:  (Look for these sections in your web.config, most likely don’t have to add the httphandler tags because they will be in there already then just add the lines inside them.)

<httpHandlers>

<add path=”/robots.txt” verb=”GET” type=”System.Web.UI.SimpleHandlerFactory” />

<add path=”*.txt” verb=”GET” type=”System.Web.StaticFileHandler” />

</httpHandlers>

<buildProviders>

<add extension=”.txt” type=”System.Web.Compilation.WebHandlerBuildProvider” />

</buildProviders>

Make sure your new robots.txt file is in your root folder for your website and everything should be good to go. Try your website out with http then https and it should be different. You will have to modify the robots.txt file to exclude what folders and files you want.

I have uninstalled Symantec Endpoint Protection from my laptop. The trial was coming to an end and with the findings of the new management console problems I didn’t see a reason to continue.

See previous posts here:

Part I

Part II

Instead I have moved on to BitDefender Total Security 2008. I have just installed it and it seems to be running ok. Total system scan was faster than I am used to with Symantec and McAfee. Setup was easy for the most part, prompted me for every program to allow it through the firewall instead of knowing most of the common programs that should be allowed like Symantec. We will see how things go, if anyone has any comments on this product please post here.

One problem that anyone who runs a server room or a datacenter has come across is how do I keep it cool. Servers put off a ton of heat, now with the popularity of 1U servers and Blade Servers the heat problem has only gotten worse.

The best way to keep a datacenter cool is to design it correctly in the beginning. Get large enough A/C units, calculate the amount of heat being put off by your equipment. Have professional cooling technicians come in and design the perfect solution. That is fine if you are designing a brand new datacenter but what do you do if you have an old datacenter that has grown beyond its cooling capability.

First off you should do what you can to direct the hot and cool air in your server room/datacenter. You should try to setup cool rows and hot rows, you don’t want to be kicking hot air from one rack into the intakes of servers in another rack. Always exhaust hot air away from all other equipment.

Next do not cool the hot air directly. It is very common to put a cold air outlet vent right over the exhaust side of your server racks. You are trying to cool the hot air before it gets to the rest of the room. The problem with this is it isn’t very efficient. The better way to cool your datacenter is to vent the cold air into the space the server intakes are drawing from. You then want to put your A/C unit’s hot air intakes right above your rack’s exhaust side. What you are doing here is sucking as much of the hot air as you can out of the room as soon as it comes out of the rack, you are then putting the cool air in place for the servers to take it in. This works well because you are cooling the ambient temperature of the room and getting rid of the hot air instead of wasting all your energy cooling really hot air. Also with the servers pulling in colder air the servers should stay cooler even if the hot rows warm up a lot.

Try it out, it works quite well when you are on a tight budget.