Unix to Windows Porting Dictionary for HPC

Porting Tips

High-Performance Computing (HPC) Resources

Microsoft has provided a great deal of resources to assist developers working with High Performance Computing (HPC) applications. These include:

Windows HPC
HPC Pack 2008 SDK with Service Pack 1 (SP1)
HPC Resource Kit
UNIX Custom Application Migration Guide (particularly Volume 3)
Windows HPC survival guide

Additionally, the UNIX Migration Project Guide discusses many of the details of moving to the Windows platform from other operating systems.

Developers using the PGI compiler may find the PGI User Guide useful, as well as information on Porting WRF to Microsoft Windows Using PGI Workstation.

Platform-specific Compiling

When modifying code during porting, it is almost always desirable to make changes in such a way that the code continues to build on the original platform. The usual way to do this is to use preprocessor conditionals for the compiler; directives such as #ifdef and #ifndef. Be careful, however, and make sure your conditionals test for operating system or compiler as appropriate. Just because a code is being built on Windows (detectable with "#ifdef _WIN32") doesn't mean that the changes specific to Visual Studio need to be used; the code might be compiling on Windows, but using GCC through Cygwin or MinGW.

To test for Visual Studio, use the _MSC_VER macro; for example,

// If we are building with MSVC, include the Windows headers;
// otherwise, use the POSIX unistd.h header
#ifdef _MSC_VER
  #include <windows.h>
#else
  #include <unistd.h>
#endif

A complete list of macros defined by Visual Studio 2008 can be found on MSDN at http://msdn.microsoft.com/en-us/library/b0084kay.aspx.

There are several tools one can use to handle the details of the cross-platform build process. Kitware's CMake from Kitware is an excellent tool for creating and managing cross-platform makefiles and workspaces. Qt from Nokia is another popular library used for cross-platform development.

unistd.h

Many applications written for Unix include the header file "unistd.h"; this header provides access to the POSIX API. Since many versions of Windows does not (by default) include POSIX compatibility, this header file is a good "red flag" when developers are examining source code during the porting process. If a source file includes unistd.h, it will almost certainly require modification to build with Visual Studio.

Threading

The different thread models between Unix and Windows is one of the largest hurdles to overcome when porting code. The Dictionary has entries for the most common POSIX thread functions, and details on how to do the same tasks in Windows. Additionally, IBM's developerWorks has a good example of how threaded code can be written in a cross-platform manner that will work in both Windows and Unix environments. Microsoft has some useful documentation about Multithreading with C and Win32 as well.

Environment Variables

Unix and Windows handle environment variables quite differently, and this can be a cause of confusion and frustration for developers porting software between platforms. Windows has two types of environment variables: system and user. The system variables can only be set by users with administrator access; in contrast, user variables are set by users and are contained in their profiles. The total environment is the union of these two, with some special rules for variables such as %PATH%.

The overall %PATH% variable used by Windows is constructed by combining the user and system %PATH% variable, by putting the system %PATH% first, and then the user %PATH%. This means that users cannot override %the system %PATH% setting. For developers dealing with programs that require %or expect certain environment variables, the best strategy is to never put anything in the system's %PATH% that users might want to override. In contrast to the way many Unix systems are set up, administrators cannot set up default paths and allow users to override them if needed.

The setx application allows users to create or modify environment variables, and is available on most versions of Windows.

Shared Libraries and DLLs

Using DLLs can be confusing to developers used to working with shared libraries in a Unix environment. Thankfully, there are several good tutorials available on the Web to help programmers come up to speed quickly on creating and linking to DLLs, including:

File Permissions and Ownership

Windows NT and derivatives don't use file permissions the way developers accustomed to programming in Unix may expect; instead, they use Access Control Lists (ACLs). ACLs are more powerful than the Unix-style read/write/execute designations for user/group/other, but they are also more complex. Some resources discussing ACLs on Windows in greater detail can be found at

Side-by-Side Configuration Error

One perplexing error occasionally encountered when building HPC software with Visual Studio is a "side-by-side" or SxS error. This type of error is caused by a mismatch in DLLs the application is attempting to find, and looks like the following:

Error (14001) The application has failed to start because its
side-by-side configuration is incorrect. Please see the application
event log for more detail.

If you do look in the event log, you'll see something like the following:

Activation context generation failed for "C:\Program Files (x86)\foo\foo.exe".
Dependent Assembly
Microsoft.VC80.CRT,processorArchitecture="x86",publicKeyToken="1fc8b3b9a1e18e3b",type="win32",version="8.0.50727.4053"
could not be found. Please use sxstrace.exe for detailed diagnosis.

The best way to resolve SxS errors involving MSVC dependencies is to create a setup project with a merge module. This approach is more reliable than attempting to copy binaries and edit manifests.

To create a setup project with a merge module in Visual Studio and deploy it on an a cluster running HPC Server, perform the following steps:

In Visual Studio, create a new Setup and Deployment project. (This differs from say, a Web project or C++ project. It's a separate category, and is often overlooked or ignored.)
Right Click on the Project and select "Add Merge Module". You should be given a list of merge modules available in your version of Visual Studio. Dependencies should be automatically resolved.
Build the project. This will create a setup.exe and a .msi file. You don't need the setup.exe; copy the .msi onto a share accessible by the node that doesn't have Visual Studio and run it. You can double-click on the MSI or, if you have many nodes, you can run the setup unattended from the command line with:
```
msiexec /qn /i \\path\to\file.msi
```

Once you do this, if you look at "Programs and Features" in the Control Panel on the new node, you'll see the setup project you created installed. You can remove it by clicking on it if it doesn't resolve the problem. (Another benefit of using this setup/merge module procedure is that it makes it easy to check when nodes you have updated, and you undo things cleanly if need be.)

Sometimes you can still have problems if the executable you are trying to build wasn't built using Visual Studio, or if Visual Studio was upgraded between the time you created the target executable and the time you create the setup/merge module project described. In that case, you may need to rebuild your projects.