Update readme, rarrange combozzle a bit for config/experiment conveni…

…ence

README.md

@@ -1,48 +1,44 @@
 # drivers_bozzle
 
-Scalable test case
+Scalable prediction-adjacent testing application.  The drivers in this
+driver suite are intended to exercise all of the features used in 
+the _MIRGE-Com_ prediction runs at CEESD.
 
-Bozzle (bozzle.py) is a production-adjacent performance testing driver
+Combozzle (combozzle.py) is a production-adjacent performance testing driver
 that is constructed by using a simple box mesh generator in place of
-the production driver (nozzle.py) gmsh-based grid generator. The bozzle
-driver, like the production drivers, requires the production branch of
-MIRGE-Com in order to run.
-
-By default, it uses order 1 elements, RK4, Navier-Stokes, and artificial 
-viscosity. It is a single air-like gas (i.e. no mixture species).  The
-solution init, bcs, and time-dependent injection features match those used
-by the production driver nozzle.py.
-
-This is all set up to run on Lassen. The *bsub* scripts are designed to
-be submitted with `bsub <script>` on Lassen.  
---------
-eager_weak_bsub.sh: Run an eager weak scaling test
-grid_scale_{eager,lazy}_bsub.sh: Run {eager, lazy} grid scaling test
-
-The support scripts are as follows:
---------
-3 supporting scripts used by the *bsub* scripts.
-run_eager_weak.sh: loop over orders and scales for weak MPI scaling with eager
-run_order_scale_{eager,lazy}.sh: loop over orders and grid scales
-
-2 scripts to run MIRGE-Com itself used by all/most others:
-run_mirgecom.sh: Run on one GPU
-run_mirgecom_dist.sh: Distribute to multiple GPUs with MPI
-
-1 script to generate an input YAML file read by bozzle.py:
-generate_input_yaml.sh
-
-several scripts for post-processing the data into sqlite/yaml files:
-create_db_summary.sh: runs runalyzer to gather sqlite data
-generate_yaml_run_file.sh: create run-specific yaml data
-extract_timing_data.sh: extracts the run timing data from sqlite into global yaml
-
-Several yaml-input-based options have been added for use in exploring
-the behavior of MIRGE-Com:
-
-wscale: Grid scaling param (nelem approx = 384*scale) default[1]
-hScaling: Scale physical box extent by *weak_scale* (0) , or not (1) default[1] 
-gridOnly: generate the grid and (0) step, or (1) exit default[0]
-discrOnly: generate the grudge discretization and (0) step, or (1) exit default[0]
-initOnly: exit after initialization (1) or continue to stepping (0) default[0]
-boundaryReport: generate a report abt npts/boundary (1), or not (0) default[0]
+the production driver (isolator.py) gmsh-based grid generator. The combozzle
+driver, like the production drivers, requires the `production` branch of
+_MIRGE-Com_ in order to run.
+
+The `combozzle.py` driver is the main driver.  The features and 
+(optional) features are roughly as follows:
+
+- MPI: 1 rank (n ranks)
+- DT: fixed (fixed cfl)
+- Timestepping: Euler (rk4, rk54)
+- RHS: Navier-Stokes (Euler, dummy)
+- EOS: Mixture (nspecies inert, single gas)
+- Shock cap: Laplacian AV (off)
+- Additional: Sponge (off)
+
+All features used by _MIRGE-Com_ in the CEESD prediction runs
+are ON by default.  The many parameters for the configuration
+of features and run details can be found near the top of the 
+`main` function in `combozzle.py`.  The code can also read config
+parameters from YAML files.
+
+--------------------
+
+Most of the subdirectories contain experiments  set up to run on Lassen@LLC.
+The *bsub* scripts are designed to be submitted with `bsub <script>` on Lassen.
+Further descriptions for each experiment should be found in README in the 
+experiment-specific directories.
+
+--------------
+
+Local files and directories:
+
+combozzle.py: the main driver
+grid_scale: directory containing grid-scaling experiments
+scripts: a parking area for scripts used to build up experiments
+config: sample yaml input

combozzle.py

@@ -174,34 +174,16 @@ def main(ctx_factory=cl.create_some_context, use_logmgr=True,
     from mirgecom.simutil import global_reduce as _global_reduce
     global_reduce = partial(_global_reduce, comm=comm)
 
-    logmgr = initialize_logmgr(use_logmgr,
-        filename=f"{casename}.sqlite", mode="wu", mpi_comm=comm)
-
-    if use_profiling:
-        queue = cl.CommandQueue(cl_ctx,
-            properties=cl.command_queue_properties.PROFILING_ENABLE)
-    else:
-        queue = cl.CommandQueue(cl_ctx)
-
-    if lazy:
-        actx = actx_class(comm, queue, mpi_base_tag=12000,
-                allocator=cl_tools.MemoryPool(cl_tools.ImmediateAllocator(queue)))
-    else:
-        actx = actx_class(comm, queue,
-                allocator=cl_tools.MemoryPool(cl_tools.ImmediateAllocator(queue)),
-                force_device_scalars=True)
-
     # Some discretization parameters
     dim = 3
-    # nel_1d = 8
     order = 1
     x_scale = 1
     y_scale = 1
     z_scale = 1
-    chlen = .25
     domain_xlen = 1.
     domain_ylen = 1.
     domain_zlen = 1.
+    chlen = .25  # default to 4 elements/axis = x_len/chlen
 
     # {{{ Time stepping control
 
@@ -228,7 +210,7 @@ def main(ctx_factory=cl.create_some_context, use_logmgr=True,
 
     # }}}  Time stepping control
 
-    init_temperature = 1500.
+    init_temperature = 700.
     init_pressure = 101325.
     init_density = 1.0
 
@@ -247,7 +229,6 @@ def main(ctx_factory=cl.create_some_context, use_logmgr=True,
 
     n_refine = 1
     weak_scale = 1
-    periodic = (periodic_boundary == 1,)*dim
 
     # av parameters
     alpha_sc = 0.5
@@ -539,6 +520,7 @@ def main(ctx_factory=cl.create_some_context, use_logmgr=True,
         box_ll = (xleft, ybottom, zback)
         box_ur = (xright, ytop, zfront)
 
+    periodic = (periodic_boundary == 1,)*dim
     if rank == 0:
         print(f"---- Mesh generator inputs -----\n"
               f"\tDomain: [{box_ll}, {box_ur}], {periodic=}\n"
@@ -549,9 +531,22 @@ def main(ctx_factory=cl.create_some_context, use_logmgr=True,
 
     wall_temperature = init_temperature
     temperature_seed = init_temperature
-
     debug = False
 
+    if use_profiling:
+        queue = cl.CommandQueue(cl_ctx,
+            properties=cl.command_queue_properties.PROFILING_ENABLE)
+    else:
+        queue = cl.CommandQueue(cl_ctx)
+
+    if lazy:
+        actx = actx_class(comm, queue, mpi_base_tag=12000,
+                allocator=cl_tools.MemoryPool(cl_tools.ImmediateAllocator(queue)))
+    else:
+        actx = actx_class(comm, queue,
+                allocator=cl_tools.MemoryPool(cl_tools.ImmediateAllocator(queue)),
+                force_device_scalars=True)
+
     rst_path = "restart_data/"
     rst_pattern = (
         rst_path + "{cname}-{step:04d}-{rank:04d}.pkl"
@@ -629,6 +624,11 @@ def vol_max(x):
 
     vis_timer = None
 
+    casename = f"{casename}-d{dim}p{order}e{global_nelements}n{nparts}"
+
+    logmgr = initialize_logmgr(use_logmgr,
+        filename=f"{casename}.sqlite", mode="wu", mpi_comm=comm)
+
     if logmgr:
         logmgr_add_cl_device_info(logmgr, queue)
         logmgr_add_device_memory_usage(logmgr, queue)
@@ -1093,6 +1093,8 @@ def dummy_rhs(t, state):
     elif use_profiling:
         print(actx.tabulate_profiling_data())
 
+    comm.Barrier()
+
     finish_tol = 1e-16
     assert np.abs(current_t - t_final) < finish_tol