Building Caffe on Windows with CUDA 9.0 for VS 2013, 2015 and 2017

Building Caffe on Windows 10 has been a journey (to put it lightly). Since I have a new gen gfx card (new for 2018), it does not support CUDA 8.0. So, all posts saying you need CUDA 8.0 are outdated (at least to me). With CUDA 9.0, Windows caffe does not compile.

The good news after a couple of days of trying I’ve figured out a workaround. The only boost version that supports CUDA 9.0 as of now is boost 1.65.1 and above. But, interestingly cmake breaks with boost 1.66.0. I know, welcome to the real-world versioning hell when it comes to actually building stuff.

So, if you got your windows source from: https://github.com/BVLC/caffe/tree/windows

You need to do a couple of stuff. First download and install boost 1.65.1 in some path. Let’s call this root directory my_boost_1_65_1 (typically C:\local\boost_1_65_1), and the library directory (which changes based upon which VS version you downloaded, typically C:\local\boost_1_65_1\lib64-msvc-14.0 for VS 2015). Yes, it sucks that MSVC version is 14.0 for VS 2015, but such is the life living in a Microsoft world.

I assume you have checked out the 1.0 version of windows caffe. Now, open the build_windows.cmd in the scripts directory, and modify the cmake command as follows (note the 3 lines referring to boost and boost paths):

cmake -G"!CMAKE_GENERATOR!" ^
-DBLAS=Open ^
-DCMAKE_BUILD_TYPE:STRING=%CMAKE_CONFIG% ^
-DBUILD_SHARED_LIBS:BOOL=%CMAKE_BUILD_SHARED_LIBS% ^
-DBUILD_python:BOOL=%BUILD_PYTHON% ^
-DBUILD_python_layer:BOOL=%BUILD_PYTHON_LAYER% ^
-DBUILD_matlab:BOOL=%BUILD_MATLAB% ^
-DCPU_ONLY:BOOL=%CPU_ONLY% ^
-DCOPY_PREREQUISITES:BOOL=1 ^
-DINSTALL_PREREQUISITES:BOOL=1 ^
-DUSE_NCCL:BOOL=!USE_NCCL! ^
-DCUDA_ARCH_NAME:STRING=%CUDA_ARCH_NAME% ^
-DBOOST_NO_BOOST_CMAKE=TRUE ^
-DBOOST_ROOT=C:/boost_1_65_1 ^
-DBOOST_LIBRARYDIR=C:/boost_1_65_1/lib64-msvc-14.0 ^
"%~dp0\.."

I do the following edit as well, so I know that boost version is correct or it will fail. So, edit the boost version in Dependencies.cmake (located in the cmake directory):


# ---[ Boost
find_package(Boost 1.65 REQUIRED COMPONENTS system thread filesystem)
list(APPEND Caffe_INCLUDE_DIRS PUBLIC ${Boost_INCLUDE_DIRS})
list(APPEND Caffe_DEFINITIONS PUBLIC -DBOOST_ALL_NO_LIB)
list(APPEND Caffe_LINKER_LIBS PUBLIC ${Boost_LIBRARIES})

And, by changing these two files, everything should be good. Now, execute scripts\build-windows.cmd and watch your build succeed.


l_prerequisites.vcxproj]
CUSTOMBUILD : warning : cannot resolve item 'api-ms-win-crt-heap-l1-1-0.dll' [D:\tools\caffe\build\tools\upgrade_solver_proto_text_install_prerequisites.vcxproj]
CUSTOMBUILD : -- warning : gp_resolved_file_type non-absolute file 'api-ms-win-crt-heap-l1-1-0.dll' returning type 'system' -- possibly incorrect [D:\tools\caffe\build\tools\upgrade_solver_proto_text_install_
prerequisites.vcxproj]
CUSTOMBUILD : warning : cannot resolve item 'api-ms-win-crt-stdio-l1-1-0.dll' [D:\tools\caffe\build\tools\upgrade_solver_proto_text_install_prerequisites.vcxproj]
CUSTOMBUILD : -- warning : gp_resolved_file_type non-absolute file 'api-ms-win-crt-stdio-l1-1-0.dll' returning type 'system' -- possibly incorrect [D:\tools\caffe\build\tools\upgrade_solver_proto_text_install
_prerequisites.vcxproj]
CUSTOMBUILD : warning : cannot resolve item 'api-ms-win-crt-convert-l1-1-0.dll' [D:\tools\caffe\build\tools\upgrade_solver_proto_text_install_prerequisites.vcxproj]
CUSTOMBUILD : -- warning : gp_resolved_file_type non-absolute file 'api-ms-win-crt-convert-l1-1-0.dll' returning type 'system' -- possibly incorrect [D:\tools\caffe\build\tools\upgrade_solver_proto_text_insta
ll_prerequisites.vcxproj]

6897 Warning(s)
0 Error(s)

Time Elapsed 00:14:44.49

Let me know in the comments if you run into more issues.

Solving the Error 127 from Make when cross compiling to raspberry pi

I was following the excellent tutorial from Alessio on cross compiling to raspberry pi from Windows 7. Then, I hit the the dreaded ‘Error 127’ from make. Now, after hours of searching I couldn’t find how to solve this. Then, Bryan has mentioned that you need to install the 32 bit cygwin version and that would work, and works it does.

If you already installed cygwin 64 bit version like me and wondering how to install it,here are some steps:

  1. download the 32 bit setup to a new directory(ex: C:\cygwin32) and run the setup. Make sure you make every path point to this new directory during the setup process.
  2. Now, copy the crosstools directory to this new directory. So, it will be at C:\cygwin32\opt\cross\x-tools\arm-unknown-linux-gnueabi
  3. Change all paths created in Eclipse in the ‘Path’ variable (and Windows Environment variables too, if you added cygwin there) to point to this new location
  4. Now, clean and build the project and it should compile for you!

Another error I ran across is this:

Could not determine GDB version after sending: C:\cygwin32\opt\cross\x-tools\arm-unknown-linux-gnueabi\bin\arm-unknown-linux-gnueabi-gdb.exe –version

If you ran across this you installed python 2.7 instead of 2.6. Re-run the cygwin setup and change the python version to 2.6.x.x and install this python version. Re-run your program and it should be working.

I wrote this so it helps someone else to save a ton of time. HTH.

HOWTO: Run Apache Hive on Windows in 6 easy steps

Note: You need to have cygwin installed to run this tutorial, as Hadoop (needed by Hive) needs cygwin to run on windows. At a minimum, Basic, Net (OpenSSH,tcp_wrapper packages) and Security related Cygwin packages need to be present in the system.

Here are the 6 steps:

1. Download WSO2 BAM 2.0.0. It’s free and open source.

2. Extract it to a preferred location. Let’s call it $BAM_HOME.

3.Start the server by executing the wso2server.bat file present in $BAM_HOME/bin. The server would startup on the default port 9443 on the machine’s IP.

4. Log in to the web console at https://localhost:9443 using the default credentials, i.e. username: admin, password: admin and clicking “Sign-In”.

WSO2 BAM login screen
WSO2 BAM login screen

5. Navigate to the “Add Analytics” option by clicking the menu item on the left hand menu.

WSO2 BAM left hand menu - add analytics option
WSO2 BAM left hand menu – add Analytics option

6. Now execute your Hive script, by entering the script and clicking execute!

Note: Follow this KPI sample document to see a sample working for you in no time, with results appearing on a dashboard. Also, notice that you can schedule the Hive script as well.

Execute Apache Hive script
Execute Apache Hive script

I have to thank my colleague Buddhika Chamith, as all this was possible because of some grueling work done by him. Also, I hate the fact Hadoop and Hive makes it so hard to run stuff on Windows, especially since this is a Java application. Read about those concerns here.

YWHSJMGWZ8BA