Convenience codes for MESA

[earlbellinger]

As just discussed in our most recent mesadev meeting, I would like to raise the idea of bundling a convenience shell script with MESA that provides useful command-line functionality to users.

The main ones I currently have in mind are:

• mesa_cp: copies a MESA working directory but excludes LOGS, photos, .mesa_temp_cache
• mesa_sed: alters a parameter in an inlist from the command line, e.g. mesa_sed inlist_start initial_mass 1.5
• mesa_grep: searches through the MESA source for matching lines of code
• mesa_zip: bundles the inlists of a working directory into a zip file for distribution
• mesa_defaults: copy the profile_columns and history_columns directories to the current directory, also followed by an optional list of arguments saying which variables to uncomment (e.g. mesa_defaults nu_max Delta_nu would uncomment those two lines)

This script could live somewhere in the MESA source tree (where? mesa/scripts? Rob suggested avoiding mesa/utils), for example named as shmesa.sh, and we could encourage users to add source shmesa.sh to their .bashrc during installation.

Two devs (Warrick in private communication, and Bill during the meeting) suggested these to live in mesa_contrib, but Bill seemed to change his mind during the meeting to thinking that it should rather live in the main MESA branch. So I think it's also worth discussing whether this is appropriate for the main branch or not.

On a related note, stash.py currently lives in star/work which does not seem to be the most appropriate place for it. Maybe we can think about a new directory to place both of these scripts? What do people think?

[warrickball]

My view would soften if there were one command with subcommands like cp, sed, grep, zip. This could also just be a separate pip installable Python package that provides one entrant script. I think users would benefit from the script ending up somewhere naturally in their path (though we already force them to set environment variables).

I've realised @wmwolf already has a Ruby tool like this:

https://billwolf.space/mesa_cli/

What would be the relationship with that?

[warrickball]

mesa sed might already be covered by f90nml but I agree is useful.

mesa grep string doesn't seem more useful than grep string $MESA_DIR/*/*/*.f90 or similar. I think once there were enough options in mesa grep to get the behaviour I sought, it'd be most of the way to just invoking grep directly. I'm open-minded though and might just not grep through the code like others.

As a counter to my own increasing curmudgeonliness, if you would find such a tool useful, others might too, so if you can implement it neatly go ahead, and we'll just see if others find it useful!

[earlbellinger]

I have no objection to removing the underscores. This is pretty straightforward to do in bash with a little bit of parsing.

[earlbellinger]

Here's a first draft for a collection and driver for common MESA tasks, written in pure bash.

The main goal is to be simple and small while providing a number of convenience functions for MESA users.

To that end, all you would need to do is add source $MESA_DIR/scripts/shmesa.sh to your ~/.bashrc and then you can do:

> mesa
      _     __  __ _____ ____    _
  ___| |__ |  \/  | ____/ ___|  / \
 / __| '_ \| |\/| |  _| \___ \ / _ \
 \__ \ | | | |  | | |___ ___) / ___ \
 |___/_| |_|_|  |_|_____|____/_/   \_\

Usage: mesa [work|change|defaults|cp|grep|zip|version|update|help] [arguments]

Subcommands:
  work      copy the work directory to the current location
  change    change a parameter in the given inlist
  defaults  copy the history/profile defaults to the current location
  cp        copy a MESA directory without copying LOGS, photos, etc.
  grep      search the MESA source code for a given string
  zip       prepare a MESA directory for sharing
  help      display this helpful message
  -h        get additional details about any of the above

Want more details about any of the subprograms? Just run any of them with -h, for example

> mesa cp -h
Usage: mesa cp source_dir target_dir
Copies a MESA working directory but without copying LOGS, photos, or .mesa_temp_cache

Some example usage

You can use this for example to make a grid of models, and e.g. include the output delta_nu and nu_max in LOGS with

### Make a grid in M and Z in a few lines of bash

# copy over a new work directory 
mesa work "grid_dir"
cd grid_dir
./mk

# output asteroseismic quantities in `LOGS/history.data` 
mesa defaults delta_nu nu_max

# loop through values of M and Z and run MESA 
for M in 1.0 1.5 2.0; do 
    for Z in 0.01 0.02 0.03; do 
        mesa change inlist_project \
            initial_mass $M \
            Zbase $Z \
            initial_z $Z
        ./star inlist_project 
        mv LOGS "M=${M}_Z=${Z}" 
    done 
done 

Tests

There is functionality for testing all the subprograms:

> mesa test
>> TESTING SHMESA <<

>>> test: mesa work
SHMESA> DEBUG: Calling work with arguments: shmesa_test_work
gfortran -Wno-uninitialized -fno-range-check -fmax-errors=7  -fprotect-parens -fno-sign-zero ...
<<< success


>>> test: mesa change
SHMESA> DEBUG: Calling change with arguments: inlist_project pgstar_flag .false.
SHMESA> DEBUG: SHMESA> BACKING UP: inlist_project inlist_project.bak
SHMESA> DEBUG: Calling change with arguments: inlist_project initial_mass 1.2 Z 0.01 Zbase 0.01
SHMESA> DEBUG: SHMESA> BACKING UP: inlist_project inlist_project.bak
<<< success

...

>>> test: mesa defaults
SHMESA> DEBUG: Calling defaults with arguments: nu_max Delta_nu
Warning: Parameter 'Delta_nu' not found in either profile_columns.list or history_columns.list.
SHMESA> DEBUG: Calling defaults with arguments: logM
SHMESA> DEBUG: SHMESA> BACKING UP: profile_columns.list
SHMESA> DEBUG: SHMESA> BACKING UP: history_columns.list
<<< success

all done!

Development

There's instructions inside the code for adding a new bash subprogram to this driver:

    ### DEVELOPMENT USE 
    ### TEMPLATE FOR NEW SHMESA FUNCTIONS 
    mesa_template () {
        if mesa_check_h_flag "$@"; then
            echo "Usage: mesa funcname arg [optional arg]"
            echo "put discription here"
            return 0
        fi

        UNTESTED # comment this out when done (crashes the program to avoid problems)

        # parse required variables 
        local example_var=5 
        if [[ ! -z $1 ]]; then # check if it's set 
            example_var=$1
        #else # uncomment if you want this to be a required variable
        #    echo "Error: "
        #    return 1
        fi
        #shift # and then copy this block to parse the next one, for example 

        # afterwards, update:
        # 1. mesa_test 
        # 2. mesa_help 
        # 3. parser at the end of this file 
    }

I'm currently developing it here. Not all the functionality is implemented yet; I wanted to discuss with everybody and get more input from you all first before proceeding further.

Discussion

As Warrick points out, it is largely a rewrite of Bill's MESA CLI (though at least currently with less/different features). One benefit of this implementation is that it is in pure bash with no external dependencies, and therefore may make more sense to include under the main MESA branch. A side benefit of such a decision would be that it is always up to date with the exact version of MESA that it is bundled with. Also, we could recommend its use (e.g. with mesa grep) to mesa-users. It could also potentially be a driver for installing and updating MESA (e.g. mesa install, mesa version, mesa update).

Bill mentioned in the call that he's currently abandoned development of MESA CLI and that it has potentially fallen out of date (but also that he has been considering resurrecting it). Currently this program unfortunately shares the same command name (mesa) as the CLI, so we should discuss that point as well.

Therefore, I think it would be useful to bundle this code with MESA, for example under a new $MESA_DIR/scripts, and encourage users to add it to their ~/.bashrc.

What do people think?

[fxt44]

some devs & users will likely find this useful. nice that its pure shell with no dependencies (one hopes). thumbs up from me to include this tool in the core distribution.

[wmwolf]

I advocate for using this (and building on it), MESA CLI be damned. As @fxt44 points out, the dependencies are lower, and having this be available "out of the box" will encourage more usage and, thus, more support. Once this ships, I'll add a deprecation notice to the mesa CLI project page.

[earlbellinger]

Thanks, I am glad to hear your support. I will make a new branch and port it in, and make a short interactive tutorial so that you all can give feedback about the user experience. I will be on vacation for the rest of the week so I'll do that when I return; please feel free to already give any suggestions in the meantime.

[evbauer]

I like it! As mentioned during the dev meeting, I think there's a lot of utility for us as devs in having these kinds of scripts available to users without having to do any extra installation. That will allow us to send simple instructions when interacting with users on the mesa-users list, because we can assume that they have simple common mesa operations available. And I'm sure lots of users will appreciate the convenience, especially if we provide good documentation!

[earlbellinger]

I started a new branch and ported it in here: https://github.com/MESAHub/mesa/tree/shmesa/scripts/shmesa

[earlbellinger]

Suggestion from Meridith: rename so that it is invoked via shmesa instead of mesa (or a suitably chosen better name; any suggestions?)

Suggestion from Matthias: incorporate the binary stuff into the script; also implement shmesa test to copy over test cases

[mjoyceGR]

hard vote for shmesa

[evbauer]

I'm pro-shmesa

[warrickball]

I'm pro-shmesa.

Why do we have to source the script? Can it not be standalone? I was expecting something like

> bash $MESA_DIR/scripts/shmesa.sh [subcommand] [opts]

or, if it's exectuable, just

> $MESA_DIR/scripts/shmesa [subcommand] [opts]