Why is my Windows Home Server backup so slow?

The other night I was on the computer later than usual, late enough that my WHS backup started. As what I was doing was low-impact, I decided to let the backup run instead of postponing it. After a couple hours, I noticed that not only was the backup not done, but the progress bar was crawling along far more slowly than I remembered it being originally. It was time to investigate.

As a first step, I remoted into the server and pulled up taskmgr. On the Processes tab, no process (even demigrator.exe and whsbackup.exe, processes that are usually heavily worked during normal WHS activities) appeared to be using more than a few percent of the CPU at a time, but the aggregate CPU Usage listed at the bottom appeared pegged at 50%. As the server is dual-core, that meant an entire core was being sucked up by something not listed under Processes.

Experience suggested this was almost certainly due to hardware interrupts, but Task Manager doesn’t show CPU usage due to hardware interrupts explicitly. Process Explorer, however, does, and downloading and running ProcExp confirmed the abnormal hardware interrupts.

At this point I could have tracked down the precise source of the interrupts with a tool like kernrate. (Adi Oltean provides a good guide on how to use it.)   However, I decided to use a little intuitive problem solving first. Hardware interrupt issues tend be caused by two things:

  1. Broken drivers
  2. Hardware failures

While the drivers haven’t been updated in a while on that server, and may have bugs that have been fixed in newer versions, my usage patterns with the server have been pretty static. It was unlikely that I only recently uncovered a bug in the driver if I hadn’t seen it up to now. Further, with the number of hard drives in the box, I wouldn’t have been surprised that one of them was failing.

I checked the System event log, but found no recent errors that would suggest a failing disk (e.g. atapi errors). I did find errors from a few weeks ago that correlated to when I thought one of my drives was failing; I had been able to correct that by reseating the loose SATA cable and no errors appeared after that point.

Might my problem still be related to that, though? Windows Home Server, because it’s based on Windows Server 2003, generally doesn’t allow one to run in AHCI mode for the disks; I had my BIOS configured to run the disk controllers in IDE mode. The Windows IDE driver (atapi.sys) automatically adjusts the transfer mode for a channel when it runs into errors talking to the disk on that channel, using slower and slower DMA modes until it gives up and switches to PIO mode. PIO mode requires a lot of hardware interrupts to transfer data; that’s why DMA mode was introduced in the first place.

At that point, the problem was obvious. When the ATAPI driver detected the drive errors a few weeks ago, it put the channel into PIO mode in an attempt to eliminate the errors. I fixed the errors by reseating the cable, but never reset the IDE channel to which the drive was connected, so it was still in PIO mode, and thus generating a plethora of hardware interrupts. I confirmed this hypothesis in Device Manager; the Secondary IDE Channel‘s Properties->Advanced Settings dialog showed the Current Transfer Mode as ‘PIO Mode‘.

To fix it, I simply uninstalled the Secondary IDE Channel and rebooted the server to let it redetect the channel. After reboot and redetection, the Current Transfer Mode was restored to ‘Ultra DMA Mode 6‘. Sure enough, hardware interrupts dropped to next-to-nothing, and backup speeds were restored to their former glory.

Posted in Uncategorized. Tags: . Comments Off on Why is my Windows Home Server backup so slow?

How to mount ISO files from a share in Hyper-V

I’m a huge fan of Windows Home Server.  I love the easy-to-set-up image- and file-based backup system, and I love the ability to just slap in another 2TB drive if I start getting low on space.  In addition to music, movies, photos, and personal documents, I keep all the software install packages and ISOs I’ve downloaded on my server.  Further, with the user account synchronization built into WHS, I’ve been able to simplify my home network and no longer maintain any (permanent) Active Directory domains.

This setup has worked pretty well up to now.  In VMWare Workstation, when I want to build a new VM, I just mount the appropriate OS .ISO from \\server\Software\ and boot to the DVD.  Hyper-V, though, doesn’t like to mount ISOs over the network.  On a fresh Hyper-V R2 install, when I tried to mount a Win7 iso for a new guest, I was greeted with the following error:

The server encountered an error while configuring devices on testvm.  Failed to add device 'Microsoft Virtual CD/DVD Disk'.  The Machine Account 'Hyper-V Virtual Machine Management service' does not have read access to file share '\\server\soft...\en_windows_7_ultimate_x64_dvd_x15-65922.iso'.

In an Active Directory environment, there is a documented solution for this; when one is also managing the Hyper-V host remotely, additional configuration is required that involves constrained delegation.

In an environment without Active Directory (like my home network), or when the machines in question are in domains that don’t talk with each other, we need something else.  One option is to enable anonymous access to the share where the ISOs are stored.  This solution is fine for my home network, and may be feasible for other small networks where security isn’t as much of an issue.  While the instructions below are for Windows Home Server specifically, they are easily adapted to a bog-standard (non-WHS) file server.

  1. First, go to Administrative Tools->Local Security Policy.

    In Security Settings/Local Policies/Security Options, make the following changes:

    Network Access: Do not allow anonymous enumeration of SAM accounts and sharesDisabled
    Network Access: Let Everyone permissions apply to anonymous usersEnabled
    Network Access: Restrict anonymous access to Named Pipes and SharesDisabled
    Network Access: Shares that can be accessed anonymously – Add SOFTWARE (or the appropriate share) to the existing list

    In Security Settings/Local Policies/User Rights Assignment:

    Access this computer from a network – Add ANONYMOUS LOGON and Everyone if they’re not already there

  2. After closing the Local Security Settings window you’ll need to reboot the server or force application of security policy via gpupdate.
  3. Then, open up Computer Management and go to System Tools->Local Users and Groups->Groups.
    Windows Home Server creates several security groups that provide read-only and read/write access to the shares it manages.  Find which group offers Read-Only access to the share and add Everyone to this group.  On my computer, the Software share is managed by RO_8 and RW_8, so I added Everyone to the RO_8 group.
  4. While you’re in Computer Management, go to System Tools->Shared Folders->Shares.  In the properties for the appropriate share, add Everyone to the Share Permissions.

After following these steps, I was able to mount ISOs from the share successfully in the Hyper-V Manager.

Unfortunately, this solution has a caveat beyond just the security implications.  Windows Home Server likes you to manage everything through its interface.  If you’ve made changes out of band, WHS is happy to ‘fix’ them for you.  After every reboot, WHS removes the Everyone token from both the security group and from the share permissions.  This means that every time I reboot, I have to perform steps 3 and 4 again. This is frustrating enough that I’ve considered writing a script for this, but I reboot the server so rarely that I haven’t bothered.

Posted in Uncategorized. Tags: , , . Comments Off on How to mount ISO files from a share in Hyper-V