First commit

LICENSE

@@ -0,0 +1,85 @@
+Components: pytorch
+
+
+/*
+Copyright (c) 2018-     Xilinx, Inc              (Alessandro Pappalardo)
+Copyright (c) 2016-     Facebook, Inc            (Adam Paszke)
+Copyright (c) 2014-     Facebook, Inc            (Soumith Chintala)
+Copyright (c) 2011-2014 Idiap Research Institute (Ronan Collobert)
+Copyright (c) 2012-2014 Deepmind Technologies    (Koray Kavukcuoglu)
+Copyright (c) 2011-2012 NEC Laboratories America (Koray Kavukcuoglu)
+Copyright (c) 2011-2013 NYU                      (Clement Farabet)
+Copyright (c) 2006-2010 NEC Laboratories America (Ronan Collobert, Leon Bottou, Iain Melvin, Jason Weston)
+Copyright (c) 2006      Idiap Research Institute (Samy Bengio)
+Copyright (c) 2001-2004 Idiap Research Institute (Ronan Collobert, Samy Bengio, Johnny Mariethoz)
+
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+
+1. Redistributions of source code must retain the above copyright
+   notice, this list of conditions and the following disclaimer.
+
+2. Redistributions in binary form must reproduce the above copyright
+   notice, this list of conditions and the following disclaimer in the
+   documentation and/or other materials provided with the distribution.
+
+3. Neither the names of Xilinx, Facebook, Deepmind Technologies, NYU,
+   NEC Laboratories America and IDIAP Research Institute nor the names
+   of its contributors may be used to endorse or promote products derived
+   from this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
+LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+POSSIBILITY OF SUCH DAMAGE.
+*/
+
+##+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+## Mostly based on the Pytorch-Encoding source code, due MIT copyright below
+##+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+## Created by: Hang Zhang
+## ECE Department, Rutgers University
+## Email: [email protected]
+## Copyright (c) 2017
+##
+## This source code is licensed under the MIT-style license found below
+##+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+# MIT License
+
+# Copyright (c) 2017 Hang Zhang
+
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+
+# The above copyright notice and this permission notice shall be included in
+# all copies or substantial portions of the Software.
+
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
+
+Copyright 2012-2014 Deepmind Technologies
+Copyright 2018 Xilinx Inc
+Copyright 2017 Hang Zhang
+Copyright 2011-2013 NYU
+Copyright 2014, 2016 Facebook Inc
+Copyright 2001-2014 Idiap Research Institute
+Copyright 2006-2012 NEC Laboratories America

README.md

@@ -0,0 +1,35 @@
+# Pytorch Quantization
+
+## Introduction
+
+This repository implements a set of quantization strategies to be applied to supported type of layers.
+
+The code originally started from the Pytorch and ATen implementation of a fused GRU/LSTM, extracted as a CFFI extension and expanded from there.
+
+## Requirements
+Building currently requires an appropriate CUDA environment, but execution is supported on CPU as well.
+
+* Nvidia CUDA Toolkit (tested with CUDA 9.0)
+* [Pytorch](https://pytorch.org) (tested with version 0.3.1)
+
+## Installation
+
+1. Run `python build.py`
+2. Add current path to the python path: `EXPORT PYTHONPATH=/path/to/pytorch-quantization:PYTHONPATH`
+
+## Usage
+
+The following quantization modes are implemented for weights:
+
+* FP: full-precision, no quantization performed.
+* SIGNED_FIXED_UNIT: fixed point quantization between [-1,1).
+
+The following quantization modes are implemented for activations:
+
+* FP: full-precision, no quantization performed.
+* SIGNED_FIXED_UNIT: fixed point quantization between [-1,1).
+
+The following quantized layers are implemented:
+
+* QuantizedLinear
+* QuantizedLSTM 

build.py

@@ -0,0 +1,99 @@
+##+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+## Mostly based on the Pytorch-Encoding source code, due MIT copyright below
+##+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+## Created by: Hang Zhang
+## ECE Department, Rutgers University
+## Email: [email protected]
+## Copyright (c) 2017
+##
+## This source code is licensed under the MIT-style license found below
+##+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+# MIT License
+
+# Copyright (c) 2017 Hang Zhang
+
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+
+# The above copyright notice and this permission notice shall be included in
+# all copies or substantial portions of the Software. 
+
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
+import os
+import torch
+import platform
+import subprocess
+from torch.utils.ffi import create_extension
+
+lib_path = os.path.join(os.path.dirname(torch.__file__), 'lib')
+cwd = os.path.dirname(os.path.realpath(__file__))
+quantization_lib_path = os.path.join(cwd, "quantization", "lib")
+
+# clean the build files
+clean_cmd = ['bash', 'clean.sh']
+subprocess.check_call(clean_cmd)
+
+# build CUDA library
+os.environ['TORCH_LIB_DIR'] = lib_path
+if platform.system() == 'Darwin':
+    os.environ['TH_LIBRARIES'] = os.path.join(lib_path,'libATen.1.dylib')
+    QUANTIZATION_LIB = os.path.join(cwd, 'quantization/lib/libQUANTIZATION.dylib')
+
+else:
+    os.environ['TH_LIBRARIES'] = os.path.join(lib_path,'libATen.so.1')
+    QUANTIZATION_LIB = os.path.join(cwd, 'quantization/lib/libQUANTIZATION.so')
+
+build_all_cmd = ['bash', 'make.sh']
+subprocess.check_call(build_all_cmd, env=dict(os.environ))
+
+# build FFI
+sources = [
+    'quantization/src/quantized_fused_rnn_cuda_wrapper.cpp'
+    ]
+headers = [
+    'quantization/src/quantized_fused_rnn_cuda_wrapper.h'
+]
+defines = [('WITH_CUDA', None)]
+with_cuda = True 
+
+include_path = [os.path.join(lib_path, 'include'),
+                os.path.join(cwd,'quantization/kernel'), 
+                os.path.join(cwd,'quantization/kernel/include'), 
+                os.path.join(cwd,'quantization/kernel/thcunn_include'), 
+                os.path.join(cwd,'quantization/src')]
+
+def make_relative_rpath(path):
+    if platform.system() == 'Darwin':
+        return '-Wl,-rpath,' + path
+    else:
+        return '-Wl,-rpath,' + path
+
+ffi = create_extension(
+    'xilinx.torch',
+    package=False,
+    headers=headers,
+    sources=sources,
+    define_macros=defines,
+    relative_to=__file__,
+    with_cuda=with_cuda,
+    include_dirs = include_path,
+    extra_link_args = [
+        make_relative_rpath(lib_path),
+        make_relative_rpath(quantization_lib_path),
+        QUANTIZATION_LIB,
+    ],
+)
+
+if __name__ == '__main__':
+    ffi.build()

clean.sh

@@ -0,0 +1,3 @@
+#!/usr/bin/env bash
+
+rm -rf build/ dist/ quantization.egg-info/ quantization/lib/ quantization/_ext/ __pycache__ quantization/__pycache__

make.sh

@@ -0,0 +1,5 @@
+#!/bin/bash
+
+mkdir -p quantization/lib && cd quantization/lib
+cmake ..
+make

quantization/CMakeLists.txt

@@ -0,0 +1,100 @@
+##+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+## Mostly based on the Pytorch-Encoding source code, due MIT copyright below
+##+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+## Created by: Hang Zhang
+## ECE Department, Rutgers University
+## Email: [email protected]
+## Copyright (c) 2017
+##
+## This source code is licensed under the MIT-style license found below
+##+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+# MIT License
+
+# Copyright (c) 2017 Hang Zhang
+
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+
+# The above copyright notice and this permission notice shall be included in
+# all copies or substantial portions of the Software. 
+
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
+CMAKE_MINIMUM_REQUIRED(VERSION 2.8 FATAL_ERROR)
+CMAKE_POLICY(VERSION 2.8)
+
+INCLUDE(${CMAKE_CURRENT_SOURCE_DIR}/cmake/FindTorch.cmake)
+
+IF(NOT CUDA_FOUND)
+FIND_PACKAGE(CUDA 6.5 REQUIRED)
+ENDIF()
+
+##+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+## Boilerplate compiler flags
+##+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+
+# Detect CUDA architecture and get best NVCC flags
+IF(NOT COMMAND CUDA_SELECT_NVCC_ARCH_FLAGS OR MSVC)
+  INCLUDE(${CMAKE_CURRENT_SOURCE_DIR}/cmake/select_compute_arch.cmake)
+ENDIF()
+LIST(APPEND CUDA_NVCC_FLAGS $ENV{TORCH_NVCC_FLAGS})
+CUDA_SELECT_NVCC_ARCH_FLAGS(NVCC_FLAGS_EXTRA $ENV{TORCH_CUDA_ARCH_LIST})
+LIST(APPEND CUDA_NVCC_FLAGS ${NVCC_FLAGS_EXTRA})
+
+if(CMAKE_CXX_COMPILER_ID STREQUAL "GNU")
+  if(CMAKE_CXX_COMPILER_VERSION VERSION_GREATER "4.9.3")
+    if(CUDA_VERSION VERSION_LESS "8.0")
+      MESSAGE(STATUS "Found gcc >=5 and CUDA <= 7.5, adding workaround C++ flags")
+      set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -D_FORCE_INLINES -D_MWAITXINTRIN_H_INCLUDED -D__STRICT_ANSI__")
+    endif(CUDA_VERSION VERSION_LESS "8.0")
+  endif(CMAKE_CXX_COMPILER_VERSION VERSION_GREATER "4.9.3")
+endif(CMAKE_CXX_COMPILER_ID STREQUAL "GNU")
+
+IF(MSVC)
+  LIST(APPEND CUDA_NVCC_FLAGS "-Xcompiler /wd4819")
+  ADD_DEFINITIONS(-DTH_EXPORTS)
+ENDIF()
+
+SET(CMAKE_MACOSX_RPATH 1)
+SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11")
+
+##+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+## Actual compilation stuff
+##+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+
+FILE(GLOB kernel_src kernel/*.cu)
+
+MESSAGE(STATUS "TORCH_INCLUDE_DIR:" ${TORCH_INCLUDE_DIR})
+MESSAGE(STATUS "TORCH_TH_INCLUDE_DIR:" ${TORCH_TH_INCLUDE_DIR})
+MESSAGE(STATUS "TORCH_THC_INCLUDE_DIR:" ${TORCH_THC_INCLUDE_DIR})
+
+
+CUDA_INCLUDE_DIRECTORIES(
+	${CMAKE_CURRENT_SOURCE_DIR}/kernel
+	${CMAKE_CURRENT_SOURCE_DIR}/kernel/include
+    ${CMAKE_CURRENT_SOURCE_DIR}/kernel/thcunn_include    
+    ${TORCH_INCLUDE_DIR}
+    ${TORCH_TH_INCLUDE_DIR}
+    ${TORCH_THC_INCLUDE_DIR} 
+)
+
+CUDA_ADD_LIBRARY(QUANTIZATION SHARED ${kernel_src})
+
+IF(MSVC)
+  SET_TARGET_PROPERTIES(QUANTIZATION PROPERTIES PREFIX "lib" IMPORT_PREFIX "lib")
+ENDIF()
+
+TARGET_LINK_LIBRARIES(QUANTIZATION 
+	${THC_LIBRARIES} 
+	${TH_LIBRARIES} 
+)

quantization/__init__.py

@@ -0,0 +1 @@
+from quantization.modules.rnn import QuantizedGRU, QuantizedLSTM

quantization/cmake/FindTorch.cmake

@@ -0,0 +1,43 @@
+##+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+## Created by: Hang Zhang
+## ECE Department, Rutgers University
+## Email: [email protected]
+## Copyright (c) 2017
+##
+## This source code is licensed under the MIT-style license found below
+##+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+# MIT License
+
+# Copyright (c) 2017 Hang Zhang
+
+# Permission is hereby granted, free of charge, to any person obtaining a copy
+# of this software and associated documentation files (the "Software"), to deal
+# in the Software without restriction, including without limitation the rights
+# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+# copies of the Software, and to permit persons to whom the Software is
+# furnished to do so, subject to the following conditions:
+
+# The above copyright notice and this permission notice shall be included in
+# all copies or substantial portions of the Software. 
+
+# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+# SOFTWARE.
+
+# Set the envrionment variable via python
+SET(TORCH_LIB_DIR "$ENV{TORCH_LIB_DIR}")
+MESSAGE(STATUS "TORCH_LIB_DIR: " ${TORCH_LIB_DIR})
+
+# Find the include files
+SET(TORCH_TH_INCLUDE_DIR "${TORCH_LIB_DIR}/include/TH")
+SET(TORCH_THC_INCLUDE_DIR "${TORCH_LIB_DIR}/include/THC")
+SET(TORCH_ATEN_INCLUDE_DIR "${TORCH_LIB_DIR}/include/THC")
+SET(TORCH_INCLUDE_DIR "${TORCH_LIB_DIR}/include")
+
+# Find the libs. We need to find libraries one by one.
+SET(TH_LIBRARIES "$ENV{TH_LIBRARIES}")
+SET(THC_LIBRARIES "$ENV{THC_LIBRARIES}")