Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ScaLAPACK API singletons #209

Merged
merged 2 commits into from
Jan 11, 2025
Merged

ScaLAPACK API singletons #209

merged 2 commits into from
Jan 11, 2025

Conversation

mgates3
Copy link
Collaborator

@mgates3 mgates3 commented Jan 11, 2025

Replaces the internal slate_scalapack_set_target() and similar functions with singletons TargetConfig::value() that query the environment only once. For Target, this also compares the entire string instead of just the 5th character, making it safe from buffer overrun and more robust.

@mgates3
Copy link
Collaborator Author

mgates3 commented Jan 11, 2025

With some extra debugging output added, we see each MPI rank queries the variables once, and all the variables are correctly passed:

slate> env TEST_LIBS=-lslate_scalapack_api make

slate> setenv SLATE_SCALAPACK_VERBOSE 1
slate> setenv SLATE_SCALAPACK_TARGET HostNest
slate> setenv SLATE_SCALAPACK_IB 32
slate> setenv SLATE_SCALAPACK_PANELTHREADS 4
slate> setenv SLATE_SCALAPACK_LOOKAHEAD 2

slate> mpirun -np 4 test/tester --ref y gesv
% SLATE version 2024.10.29, id 798d22a3
% input: test/tester --ref y gesv
% 2025-01-11 12:34:45, 4 MPI ranks, CPU-only MPI, 4 OpenMP threads per MPI rank
                                                                                                                                                                                         
type  origin  target  gemm     lu  trsm   go   do   A   B       n    nrhs    nb  ib    p    q  la  pt  threshold     error   time (s)       gflop/s  ref time (s)   ref gflop/s  status  
mpi 0: TargetConfig
mpi 0: VerboseConfig
mpi 0: LookaheadConfig
mpi 0: PanelThreadsConfig
mpi 0: IBConfig
mpi 1: TargetConfig
mpi 1: VerboseConfig
mpi 1: LookaheadConfig
mpi 1: PanelThreadsConfig
mpi 1: IBConfig
mpi 2: TargetConfig
mpi 2: VerboseConfig
mpi 2: LookaheadConfig
mpi 2: PanelThreadsConfig
mpi 2: IBConfig
mpi 3: TargetConfig
mpi 3: VerboseConfig
mpi 3: LookaheadConfig
mpi 3: PanelThreadsConfig
mpi 3: IBConfig
scalapack_api/scalapack_gesv.cc:122 slate_pgesv(): target N, verbose 1, la 2, pthreads 4, ib 32, grid_order C
   d    host    task  auto   PPLU  auto  col  row   1   1     100      10   384  32    2    2   1   2       1.00  5.11e-19    0.00522         0.165      0.000686         1.255  pass    
scalapack_api/scalapack_gesv.cc:122 slate_pgesv(): target N, verbose 1, la 2, pthreads 4, ib 32, grid_order C
   d    host    task  auto   PPLU  auto  col  row   1   1     200      10   384  32    2    2   1   2       1.00  5.50e-19   0.000800         7.639      0.000783         7.805  pass    
scalapack_api/scalapack_gesv.cc:122 slate_pgesv(): target N, verbose 1, la 2, pthreads 4, ib 32, grid_order C
   d    host    task  auto   PPLU  auto  col  row   1   1     300      10   384  32    2    2   1   2       1.00  4.00e-19    0.00137        14.418       0.00134        14.796  pass    
scalapack_api/scalapack_gesv.cc:122 slate_pgesv(): target N, verbose 1, la 2, pthreads 4, ib 32, grid_order C
   d    host    task  auto   PPLU  auto  col  row   1   1     400      10   384  32    2    2   1   2       1.00  3.23e-19    0.00460         9.951       0.00323        14.170  pass    
scalapack_api/scalapack_gesv.cc:122 slate_pgesv(): target N, verbose 1, la 2, pthreads 4, ib 32, grid_order C
   d    host    task  auto   PPLU  auto  col  row   1   1     500      10   384  32    2    2   1   2       1.00  2.97e-19    0.00545        16.175       0.00506        17.435  pass    

% Matrix kinds:
%  1: rand, cond unknown

% All tests passed: gesv

With an invalid target, it throws an error:

slate> setenv SLATE_SCALAPACK_TARGET Task    # invalid

slate> mpirun -np 4 ./test/tester --ref y --dim 100 gesv
% SLATE version 2024.10.29, id 27f266f0
% input: ./test/tester --ref y --dim 100 gesv
% 2025-01-11 13:18:48, 4 MPI ranks, CPU-only MPI, 4 OpenMP threads per MPI rank
                                                                                                                                                                                         
type  origin  target  gemm     lu  trsm   go   do   A   B       n    nrhs    nb  ib    p    q  la  pt  threshold     error   time (s)       gflop/s  ref time (s)   ref gflop/s  status  

Error on rank 0: Invalid target: Task in TargetConfig at scalapack_api/scalapack_slate.hh:186. (4 ranks had some error.)
   d    host    task  auto   PPLU  auto  col  row   1   1     100      10   384  32    2    2   1   2       1.00  5.87e-19     0.0359        0.0240            NA            NA  FAILED  

Previously, this would cause a buffer overflow when it accessed targetstr[4].

Invalid ib, lookahead, and panelthreads integers are set to 1:

slate> setenv SLATE_SCALAPACK_TARGET HostTask
slate> setenv SLATE_SCALAPACK_IB 0            # invalid
slate> setenv SLATE_SCALAPACK_LOOKAHEAD 0     # invalid
slate> setenv SLATE_SCALAPACK_PANELTHREADS 0  # invalid

slate> mpirun -np 4 ./test/tester --ref y --dim 100 gesv
% SLATE version 2024.10.29, id 27f266f0
% input: ./test/tester --ref y --dim 100 gesv
% 2025-01-11 13:30:32, 4 MPI ranks, CPU-only MPI, 4 OpenMP threads per MPI rank
                                                                                                                                                                                         
type  origin  target  gemm     lu  trsm   go   do   A   B       n    nrhs    nb  ib    p    q  la  pt  threshold     error   time (s)       gflop/s  ref time (s)   ref gflop/s  status  
scalapack_api/scalapack_gesv.cc:119 slate_pgesv(): target T, verbose 1, la 1, pthreads 1, ib 1, grid_order C
   d    host    task  auto   PPLU  auto  col  row   1   1     100      10   384  32    2    2   1   2       1.00  7.12e-19     0.0129        0.0668      0.000741         1.162  pass    

% Matrix kinds:
%  1: rand, cond unknown

% All tests passed: gesv

@mgates3 mgates3 merged commit 27f266f into icl-utk-edu:master Jan 11, 2025
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant