updated READMEs; now the entire bootstrap script works and Alchemist …

…compiles

CORI_README.md

@@ -14,15 +14,15 @@ make check # will run the current test suite
 ## Clone the Alchemist repo and set the ALPREFIX environment variable to where supporting libraries are installed
 ```
 bash # need to use bash
-mkdir -p $SCRATCH/alchemistSHELL
-cd $SCRATCH/alchemistSHELL
+mkdir -p $SCRATCH/alchemistbase
+cd $SCRATCH/alchemistbase
 git clone https://github.com/alexgittens/alchemist.git
 git checkout cori-version 
 source ./alchemist/setup/cori-bootstrap.sh
 
 module unload PrgEnv-intel
-module load PrgEnv-gnu gcc java cmake python boost sbt
-export ALPREFIX=$SCRATCH/alchemistSHELL/bins
+module load PrgEnv-gnu gcc java python boost sbt cray-hdf5-parallel fftw
+export ALPREFIX=$SCRATCH/alchemistbase/bins
 
 cd alchemist
 make build

README.md

@@ -1,7 +1,3 @@
-This branch uses h5spark. The NERSC version is broken as of 11/2/2017, so I'm using [my fork](https://github.com/rustandruin/h5spark), which seems to work
- still. You'll need to do an "sbt publish-local" so it is installed in your local Ivy repo on Cori, before 
- doing an sbt package of Alchemist.
-
 Alchemist is a framework for easily and efficiently calling MPI-based codes from Apache Spark.
 
 ![Platonic Alchemist Architecture](https://github.com/alexgittens/alchemist/blob/master/architecture.png)
@@ -24,7 +20,7 @@ export TMPDIR=/tmp # avoid a Mac specific issue with tmpdir length
 make # will both build and run the test suite
 ```
 
-# Installation instructions (for running locally on MacOS 10.12)
+# Installation instructions (for running locally on MacOS 10.12) (NEED TO BE UPDATED: adapt the setup/cori-bootstrap.sh script)
 
 ## Install some prereqs
 Assuming that the XCode command line tools, Homebrew, and Spark have been installed:

TODOS.md

@@ -14,3 +14,5 @@
 * need to fix kMeans on Spark and Alchemist side to use all the options passed in
 * need to fix kMeans on Alchemist side to use the seed appropriately on driver and workers
 * need to go through and uniformatize the datatype used for row/col indices (e.g. am sure in kmeans code, am using uint32_t for sampling row indices, but the rows can be longs)
+
+* bette placement of the third-party dependencies? want them to be under the alchemist/ directory for simplicity

setup/cori-bootstrap.sh

@@ -91,6 +91,7 @@ if [ "$WITH_RANDOM123" = 1 ]; then
   cp -r Random123-1.08/include/Random123 $ALPREFIX/include
 fi
 
+# Skylark
 # need to use development-v0.30 branch and patch it
 if [ "$WITH_SKYLARK" = 1 ]; then
   git clone https://github.com/xdata-skylark/libskylark.git