WSUS 3.0 INSTALL ERROR ON WINDOWS SERVER 2008 SP2 X64

This took me 4 nights to figure out how to fix the installation error and get the Windows Server Update Services (WSUS) 3.0 SP2 going on a Windows Server 2008 SP2 x64 server. I guess it must be common for people to encounter the problem I had as it appears to be the WSUS 3.0 SP2's fault that doesn't recognise x64 Windows properly.

Problem Description

Basically I was able to complete the majority of the installation process of WSUS 3.0 SP2 on the server. However, at the end of the installation when the first-run configuration wizard "microsoft.updateservices.ui.oobewizard" attempted to run, it will return error:

An unexpected error occurred, and this wizard must be closed.
You may restart the WSUS Server Configuration Wizard form the Option's page in the WSUS 3.0 console

Then if you try to run the WSUS administration console, it will show you a blank screen without any server connected. If you try to connect to the server to which you just installed WSUS, you will get the error saying the server didn't respond. Basically the server doesn't appear to have WSUS installed and running.

Checking the event log, you will see a number of error events related to the WSUS, they look like this:

Note that the selected error on the screenshot above indicates the error encountered by the WSUS Server Configuration Wizard.  if you extract the HTML portion from the event details and view it with web browser, you will see this (note that I lost the CSS by embedding the HTML error message into this web page):

I have done quite a bit of googling in order to find out what was causing the issue, and unfortunately most of the posts pointed me to the wrong direction and simply don't have answer, until I came across this one:

http://social.technet.microsoft.com/Forums/fi-FI/winserverwsus/thread/2edbaf80-7f24-472f-bc2f-81897e189bfa

My problem in fact caused only by one issue, not the same as what described as answers in the link above. That is: the WSUS application pool in IIS7 has an advanced setting "Enable 32-bit Applications" configured as "True":

Since my Windows server is a 64bit machine, possibly that setting shouldn't be set to true at the first place, but it was set by the installer I think, which might just be a bug. Anyway, I changed the setting to "false", restarted the application pool, then the WSUS Server Configuration Wizard started working straight away! What a quest for the root cause!

Resolution:

Set the "Enable 32-bit Applications" to "false" in the "advanced settings" of WSUS application pool, restart the application pool.