Cluster Setup

The audience for this section is the system administrator who will set up the cluster for FCMA.

The FCMA Toolbox runs under MPI. The main dependencies are an MPI implementation, a C++ compiler with OpenMP support, a tuned CBLAS implentation, andnifticlib 2.0 open source libraries. See the README files in the top-level deps/ folder in the source distribution for more details.

At the Princeton Neuroscience Institute we use the Intel compiler with Intel MPI and MKL libraries. This makes sense for us, since our main cluster has Xeon Phi boards installed in each node. While this configuration is tested the most, we test and supply setup files for other configurations.

The Makefile in the src/ directory includes a Make.user file that should be edited to supply, at a minimum, the location of the CBLAS implementation's headers and libraries. As long as you have mpic++ in your PATH and its libraries in your LD_LIBRARY_PATH when you type make you should have no problems compiling. A Make.user.linux file contains the settings necessary to compile a Linux binary using OpenBLASGCC, and OpenMPI, and Make.user.osx contains OS X settings.

pni_fcma typed alone supplies this help text:

Usage 1: [OMP_NUM_THREADS=n mpirun -np n -hostfile host] ./pni_fcma -d matrix_directory -m matrix_file_type -t output_file -k taskType [options]
Usage 2: [OMP_NUM_THREADS=n mpirun -np n -hostfile host] ./pni_fcma config.fcma
-d matrix directory, contains files in binary gz format, beginning with numbers of row and column
-m matrix file type, usually the extension name
-k task type, 0 for voxel selection using svm, 1 for smart distance ratio, 2 for searchlight, 3 for correlation sum, 4 for two parts correlation and test, 5 for cross validation of two parts correlation, 6 for one part activation and test, 7 for cross validation of one part activation, 8 for voxel correlation visualizarion
-t output file for task 0,1,2,3 in the voxel selection mode, input file for the same tasks in the test mode
-b block information file, if no block information file, a block information directory is required
-e block directory name, will check this if -b is not provided
-s step, the number of rows to be assigned per round, default 100
-l leave out id, the first block id that being left out, default -1, which means don't leave out anything
-h number of items that held for test, default -1, only and must get values when -l is applied
-c bool, test mode (1) or not (0), default 0
-n number of folds in the feature selection, default 0
-x the first mask file, default no mask
-y the second mask file, default no mask
-v the block id that you want to visualize the correlation, must be specified in task 8
-r the referred file of the output file of task 8, must be a 4D file, usually is the input data file
-q bool, being quiet (1) or not (0) in test mode for task type 2 4 5 6 7, default 0

The FCMA File page documents each of these input arguments and will generate an .fcma file that can be used with the pni_fcma_submit script, once MPI is working directly. Some effort will be required to adapt the particulars of the pni_fcma_submit script to your particular site, and advanced users may want to dispense with the layer of abstraction and use MPI directly. Several MPI implementations are scheduler-aware, obviating the need for a submit script.