Skip to content

Latest commit

 

History

History
116 lines (85 loc) · 5.78 KB

README.md

File metadata and controls

116 lines (85 loc) · 5.78 KB

Replication Procedure

This process describes the entire replication process for linux, WSL (Windows Subsystem for Linux), or on Windows with Git Bash. While most individual runs are fast, the paper has a large number of robustness checks and the full execution could take up to a week if done serially.

Setup Environment

Install conda and if using linux then install the Linux packages yourself directly, or else use the following script to do it automatically on Linux or Windows WSL:

bash scripts/setup_linux_environment.sh

You will need to setup Julia and compile required packages. On all platforms, you should be able to do this by running the following script (staying at the top level of the repo):

bash scripts/setup_julia_environment.sh

That process will do a package compilation and take 10-20 minutes depending on your operating system.

If you are not running those files then you will probably want to install the python and julia scripts with

pip install -r scripts/generate_paper_results/requirements.txt
julia --threads auto -e 'using Pkg; Pkg.activate("scripts"); Pkg.instantiate()'

Create a Sysimage

While not strictly required, you may create a sysimage to speed up the sampling process. Execute the following in a commandline within the main directory

julia --threads auto -e 'using Pkg; Pkg.add(\"PackageCompiler\")'
julia --project --threads auto ./deps/create_sysimage.jl

That process will take at least 30 minutes to fully compile. Note that the PackageCompiler.jl process is often broken, so this may not work with arbitrary Julia versions and package versions.

Simulate data (Optional)

The files are already generated in the /data directory. To regenerate the simulated data you can run

julia --sysimage JuliaSysimage.dll --project deps/generate_simulated_data_rbc.jl
julia --sysimage JuliaSysimage.dll --project deps/generate_simulated_data_rbc_sv.jl
julia --sysimage JuliaSysimage.dll --project deps/generate_simulated_data_rbc_frequentist.jl
julia --sysimage JuliaSysimage.dll --project deps/generate_simulated_data_sgu.jl

where you can replace the .dll with .so on linux or macos.

If you change the pseudo-true values in that file, you can also change the initial conditions for the samplers in the data/init_params directory. However, while useful for clean comparisons to dynare performance, they are not especially important in general.

Run Samplers

WARNING: This will take up to a day to run.

To run the primary experiments, execute:

bash scripts/run_samplers/baseline_experiments.sh

The other scripts are left separate to make running them in parallel easier. These run thousands of examples and may take 3-4 days to run in parallel, depending on your computer. However, you can execute in parallel on a multi-core machine to speed up the process.

bash scripts/run_samplers/rbc_1_joint_frequentist.sh
bash scripts/run_samplers/rbc_1_kalman_frequentist.sh
bash scripts/run_samplers/rbc_2_joint_frequentist.sh
bash scripts/run_samplers/rbc_1_joint_robustness.sh
bash scripts/run_samplers/rbc_1_kalman_robustness.sh
bash scripts/run_samplers/rbc_2_joint_robustness.sh

A few notes:

  • This uses the scripts/utilities.sh file has some BASH functions to enable easy execution of the samplers with various permutations on the arguments.
  • To only execute a subset of the runs, you can comment out lines in the baseline_experiments.sh file.
  • If you want to change the location of the data or the output, you can edit the $RESULTS_PATH and $DATA_PATH variables in the baseline_experiments.sh file.

Dynare

To setup:

  1. Install a recent version of Matlab (these were tested on Matlab 2021a)
  2. Install dynare from https://www.dynare.org/download/. These were tested with dynare 5.4
  3. Configure dynare in your matlab path (e.g., https://www.dynare.org/resources/quick_start/#configuring-matlab-for-dynare-on-windows on Windows)

Then, from the main folder do

bash scripts/run_dynare_samplers/baseline_experiments.sh

This may take some time

Then, you can run the following (in separate terminals if you wish, as the results aren't timed like the previous example). This may take 1-2 days

cd scripts/run_dynare_samplers
matlab -nosplash -nodesktop -r "run('rbc_1_robustness.m');exit;"
matlab -nosplash -nodesktop -r "run('rbc_2_robustness.m');exit;"
cd ../.. # go back to the main directory

Generating figures and tables

Assuming that you have the packages installed (i.e. julia --threads auto -e 'using Pkg; Pkg.activate("scripts"); Pkg.instantiate()' or equivalent) and either executed the above steps or downloaded a preexisting .replication_results and put it local to your computer, you can generate all of the figures and tables to .paper_results by running the following in Git Bash or bash on Linux:

bash scripts/generate_paper_results/generate_paper_results.sh

The script may take 10+ minutes if you include the convert_ lines. All final results are placed in the .paper_results directory.

The individual scripts this calls are:

  1. convert_frequentist_output.jl: Converts the output from the frequentist experiments into a format that can be used by the plotting scripts.
  2. convert_dynare_output.jl: Converts the output from the dynare experiments into a format that can be used by the plotting scripts consistent with the Julia chains.
  3. baseline_figures.jl: Generates all figures except for the RBC robustness examples
  4. rbc_robustness_figures.jl: Generates the RBC robustness figures
  5. baseline_tables.py: Generates all tables except for the RBC frequentist tables
  6. rbc_frequentist_tables.py: Generates the RBC frequentist tables

Constants such as the number of samples are extracted from metadata in the .replication_results directory.