Building VisIt quickly on LC compute resources

[markcmiller86]

1. get an interactive allocation (on poodle, I used sxterm 1 36 30 -p pdebug)
2. cd /var/tmp/<username> (I believe the <username> part of the path is created for you automatically by the scheduler)
   • This space will be either a ramdisk space or maybe on the node's local solid state drives if it has any. Very FAST!
   • A build will wind up requiring maybe 8 Gigabytes. So, this won't work if the node doesn't have the memory (for a ramdisk taking up that size) or local solid state drives.
3. mkdir build
4. cd build
5. Run the command to CMake VisIt there as in...

    /usr/WS1/visit/visit/thirdparty_shared/3.4.1/toss4/cmake/3.24.3/linux-x86_64_gcc-10.3/bin/cmake
   -DCMAKE_BUILD_TYPE:STRING="Debug"
   -DVISIT_CONFIG_SITE:STRING="/g/g11/miller86/visit/visit/34rc/build/../src/config-site/poodle18.cmake"
   /g/g11/miller86/visit/visit/34rc/src
   

6. Do the make make -k -j QQ (where QQ is 2-3x the number of cpus you have there. On 112 cpu nodes that might wind up requiring too much of the node's memory so back the number down a bit if so)
7. It should complete in 5-10 mins. Just copy it all back to your home dir or wherever...

   rsync -av build ~/visit/visit/34rc/.
   

8. You needn't worry about cleaning up what you leave in /var/tmp because when you exit the allocation, that gets cleaned up...I think. I mean, it should.

I just now did all these steps, compiling VisIt in < 7 minutes, copying back to home dir and then running the copied build and it all seems to work.

To build the whole 3rd party shootin match, we can probably do something similar on a per-package basis. I think we do the builds on the compute node's file systems but install to their final destinations.

[JustinPrivitera]

And everything works after you move the result somewhere? There aren't paths that are messed up when you run things?

[markcmiller86]

Well, everything is probably too strong a word. I didn't test everything. I tested a few things and, to my surprise, they worked after I did the rsync step to copy to my home dir.

[markcmiller86]

BTW...the rsync command used there for the copy is for cases where you engage this kind of workflow multiple times throughout a day or week using the same desination dir. It will copy over only the changed files and does some other "magic" to make it go fast. Better than a recursive cp.

[JustinPrivitera]

Interesting, that's good to know!