Super-Networking Blog

Sometimes you just need Secure ftp or sftp for some reason or another. Most ftp servers that are free do not include sftp, you need to pay for that functionality. There is a program that CoreFTP puts out called mini sftp server but you need to run this while logged in. If you log out of the server then sftp shuts down, does not run as a service.

Before you spend the $50-$200 to purchase a sftp application that runs as a service try this workaround. Download the mini sftp server, run the program setup the user and path. Then close it and open up scheduled tasks.

Add a new task:

-Browse for the mini sftp program

-Set you “Run as” user and password

-Go to Schedule Tab and change it to “At System Startup”

-Go to the Settings Tab and uncheck “Stop the task if it runs for” box

-Save the settings and run the task.

You should now have your sftp process going even when you aren’t logged in.

I found out an interesting quirk when backing up DFS (Distributed File System) files using Symantec Backup Exec 11D. I have a full backup setup, basically selected the entire drive to backup, finishes every night with no errors.

You go back into the job in Backup Exec and try to do a restore, what do you find? The folders that contain the DFS files are empty, 0 bytes. My first thought is what the hell is this all about. Was scared for a minute or two thinking I have no backups on tape of any of my DFS files.

I looked into it more and found that starting in Backup Exec 10d SP2 and continuing through 11d you cannot performance a backup of DFS files directly like you would normally do. You need to use the “DFSR Writer” which means using the Shadow Copy Components backup.

If you go into your server selection in Backup Exec, drill down into Shadow Copy Components -> User Data you will see the DFS info. Make sure to select these to backup your DFS files.

Things to keep in mind about backing up DFS in this manner:

-Backup Performance is greatly affected, could double the time needed for backing up those files.

-DFS Backups in this manner cannot be redirected to another location on restored like a lot of people including myself like to do when restoring files.

One workaround for the speed issue I have found but not tested is to run a Net Stop command for the DFS Replication service before the backup starts then a Net Start for that service when done.

What do you do if you want to look for combinations of word strings in Gigabytes of IIS log files but don’t want to spend any money?

You download a program called Strings.

This is a SysInternals program now owned by Microsoft that will search through text files for word strings and kick them out to you.

So I will go through an example of how you can use it and you can modify it from there.

So copy all of the IIS log files to a location, for example c:\logs

Then open up a command prompt, go to the directory where you have the strings program.

Type the command: strings.exe -s c:\logs | findstr “Your Search Text”

This will dump all the lines it finds with that search text to your command prompt window. Now take it further.

Type the command: strings.exe -s c:\logs | findstr “Your Search Text” > c:\output\log.txt

That will dump the found lines to a new text file. Either you can use that text file for whatever you want or parse it down further with the following.

Type the command: strings.exe -s c:\output | findstr “Your Search Text 2″

That will allow you to search through huge directories of IIS logs for combinations of things. If you do the same search often try dumping it into a batch file.

Thanks Goes to Steve For the Idea to Look Into Strings

First thing you are going to need is the Windows 2003 Resource Kit Tools

You then are going to want to find two programs, instsrv.exe and srvany.exe, I would recommend making a new folder on the root of your drive and pulling over a copy of those two files. So say a folder called myservice you really could name is anything you want.

Next what you need to do is register a service to run srvany.exe with the name of your choosing. Open a command prompt and go into the folder myservice.

Command would be:

instsrv.exe “My Service” c:\myservice\srvany.exe

-Switch “My Service” with name you want the service to have.

Next you need to tell srvany.exe what program you want it to run as a service.

Go into the registry using regedit find this key:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\My Service

-Add a new sub-key call Parameters under My Service

-In the Parameters key create a new string value named Application

-Set the value of it to the full path of the executable file location. Be sure you use “” around the path.

That should be it, you might have to reboot otherwise go into services.msc and start your service. You should be good to go.

Ran across something interesting this week. I was looking into performance on some of our webservers and noticed that there would be unexplained CPU spikes every once in a while. No real spikes in traffic, no additional tasks going on.

In the event logs I did find something that correlated with the CPU spikes:

Event ID: 7036

“The WinHTTP Web Proxy Auto-Discovery Service service entered the running state.”

Event ID: 7036

“The WinHTTP Web Proxy Auto-Discovery Service service entered the stopped state.”

This happens many times a day, starting and stopping. When in transition it seemed to spike the CPU for a little while.

Service Description is the following:

Implements the Web Proxy Auto-Discovery (WPAD) protocol for Windows HTTP Services (WinHTTP). WPAD is a protocol to enable an HTTP client to automatically discover a proxy configuration. If this service is stopped or disabled, the WPAD protocol will be executed within the HTTP client’s process instead of an external service process; there would be no loss of functionality as a result.

In looking into this more I found this is not needed to run IIS or anything else on your server. Only needed for your browser on your server to detect a proxy for Internet browsing. So do yourself a favor and disable this service, you will see it most likely set as manual and stopped but don’t let this fool you. Check out your event logs and if you see it starting and stopping disable it.

So if you are running Windows 2003 with IIS 6 and all of your web applications are running ASP .Net 2.0 you might want to get rid of .Net 1.1.

Well with Windows 2003 it isn’t easy or advisable to completely remove .Net 1.1 but you can unregister is from IIS having the same affect as far as your websites are concerned.

You need to open up the command prompt and go to the path .Net is installed in. Typically is C:\Windows\Microsoft.NET\Framework\v1.1.4322\

If you have other versions of .net installed you will see all the versions under Framework as well, be sure to go into the 1.1 directory.

From the command prompt in that directory type the command “aspnet_regiis.exe -u”

This will unregister this version of the framework with IIS and in turn IIS will not see it. You will want to do a reboot to make sure everything comes up clean afterwards.

Also make sure that you do not have any websites using .Net 1.1 because you will most likely mess it up when it is forced over to .Net 2.0

So I posted earlier on splitting up the robots.txt you are showing based on if the bot was connecting on http or https here. I commented at the end that if this is an existing site and Google has already indexed your https content you need to be careful. Once you block Google from reindexing your https content using the robots.txt file Google will ignore changes, but it will keep the old indexed pages.

You cannot use the URL removal tool because you can’t tell it to only remove https urls. The way you get around this is to remove the robots.txt restrictions from https and put meta tags for Google to remove the content only on our https content.

Tags you are going to want to user are:

<meta name=”ROBOTS” CONTENT=”NONE”>

and

<meta name=”GOOGLEBOT” content=”NOARCHIVE”>

This will tell Google to not index the pages and remove any old caches and indexes. This could take some time for Google to crawl all of your https content. Once everything you want out of their index is gone you can go ahead and change over to the easier https robots.txt file restrictions.

I do not have the code that allows you to set different meta tag info based on http or https. If someone has that please comment on this post. This should help out the SEO on your site. Thanks

Resource Page for You

One of the big things a lot of people are doing these days is SEO or search engine optimization. I am not planning on going over a comprehensive guide for SEO right now but as I run into them in my own job I will post them here.

One of the more recent tasks I had was setting up a different robots.txt for our http site vs our https site. It is the same content but Google indexes them as two different copies of the content. Google then penalizes your site for duplicate content. Well you setup your robots.txt file to exclude the files and directories you want from the bots but it doesn’t help the http vs http. Both protocols will use the same robots.txt file and so you will end up with duplicate content in Google’s index.

Here is how to solve this problem when you are running IIS 6.0 on a Windows 2003 server. You will need to also be running ASP .Net 2.0 for this solution to work. You might be able to get this to work on other platforms but I have not tested them. What we will be doing here is creating a dynamic robots.txt file, it is only one file but it will display different results depending on if you connect with http or https.

1) Create your robots.txt file:

<%@ WebHandler Language="C#" Class="MyNamespace.robotshandler" %>

using System;
using System.Web;

namespace MyNamespace {

    public class robotshandler: IHttpHandler {

        public void ProcessRequest (HttpContext context) {

            context.Response.ContentType = "text/plain";
            context.Response.Write("User-agent: *\n");

            if (context.Request.ServerVariables["Https"]=="off"){
                // HTTP
                context.Response.Write("Allow: /\n");
                context.Response.Write("Disallow: /MyDisallowedDirectory/\n");
            } else {
                // HTTPS
                context.Response.Write("Disallow: /");
            }
        }

        public bool IsReusable {
            get {return false;}
        }
    }
}

2) IIS needs to have .txt files passed through ASP .Net

- Open IIS and right click on your website and bring up the properties screen

- Go to Home Directory > Configuration. You will be on the Mappings Tab.

- Locate the ASPX item and click Edit - Copy the path in the Executable Field and cancel out of that window.

-Cancel

-Click “Add”

-Populate the Executable path with the value you copied in the last section

-Extension “.txt”

-Enter “GET” in the “Limit To” field

-Save and Exit

3) Need to modify web.config to process the .txt correctly

-Add the following under system.web:  (Look for these sections in your web.config, most likely don’t have to add the httphandler tags because they will be in there already then just add the lines inside them.)

<httpHandlers>

<add path=”/robots.txt” verb=”GET” type=”System.Web.UI.SimpleHandlerFactory” />

<add path=”*.txt” verb=”GET” type=”System.Web.StaticFileHandler” />

</httpHandlers>

<buildProviders>

<add extension=”.txt” type=”System.Web.Compilation.WebHandlerBuildProvider” />

</buildProviders>

Make sure your new robots.txt file is in your root folder for your website and everything should be good to go. Try your website out with http then https and it should be different. You will have to modify the robots.txt file to exclude what folders and files you want.

I have uninstalled Symantec Endpoint Protection from my laptop. The trial was coming to an end and with the findings of the new management console problems I didn’t see a reason to continue.

See previous posts here:

Part I

Part II

Instead I have moved on to BitDefender Total Security 2008. I have just installed it and it seems to be running ok. Total system scan was faster than I am used to with Symantec and McAfee. Setup was easy for the most part, prompted me for every program to allow it through the firewall instead of knowing most of the common programs that should be allowed like Symantec. We will see how things go, if anyone has any comments on this product please post here.

So I ran across something kind of interesting the other day. I was setting up Microsoft CRM 4.0. I got the server setup and the website configured but when I went to the website it prompted me for my username and password. I would type it in and the authentication would fail. I tried it in Firefox and it would work using my correct logged in credentials which is what Integrated Windows Authentication is.

So I check out my IIS settings, I have anonymous access disabled and Integrated Windows Authentication is checked. Why would Integrated Windows Authentication work in Firefox but not in IE. I go into Internet Options on my browser and select the advanced tab. There is an option “Enable Integrated Windows Authentication” and I have it enabled. I tried a couple of more times to make sure I wasn’t fat fingering my login and still get 401 authentication errors.

Well I thought just for the heck of it I would try unchecking the “Enable Integrated Windows Authentication” box in IE. I restarted my browser and bang it works, doesn’t prompt for password or anything. So I uncheck use Integrated Authentication and the Integrated Authentication starts working. Now I am really scratching my head.

I start searching through user groups on the Internet and I finally find the answer. In IE 6 and IE 7 the browser will use Integrated Windows Authentication if you have that checkbox enabled or disabled. The difference is the authentication type. With the box checked it with try Kerberos authentication first then fallback on NTLM. If you uncheck the box then it will just use NTLM. Thank you Microsoft for mislabeling that feature. The really annoying thing is if both ends support Kerberos authentication, you have that Integrated Windows Authentication box checked, and Kerberos fails it will not fail back to NTLM. The only way it will fail back to NTLM is if your website doesn’t support Kerberos.

What I ended up finding out was that Kerberos authentication was broken on my CRM server, quick workaround was to uncheck the box in IE. The problem with that is most features in CRM 4.0 don’t work on NTLM. I will post another time on how I fixed Kerberos on that server.