Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added ability to estimate mean velocity, density from SuNeRF, alongside better plotting codes. #10

Open
wants to merge 25 commits into
base: main
Choose a base branch
from
Open

Added ability to estimate mean velocity, density from SuNeRF, alongside better plotting codes. #10

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
0cb9443
Adding Debug output, PSI load commands
Marshelix Jul 31, 2023
5ed7745
Amend run_icarus
Marshelix Jul 31, 2023
7861ea6
changed output parameter for PSI data
hlamdouar Jul 31, 2023
3a5ceb2
Take absolute value of intensities
Marshelix Aug 1, 2023
6792dcc
fixed devices and added velocity plot
hlamdouar Aug 1, 2023
f42d684
added script for 3D plots of density and velocity. Only velocity work…
hlamdouar Aug 2, 2023
ec30117
Add Sun, Earth, L5 to plot, set alpha to be exponent in velocity
Marshelix Aug 2, 2023
6c039bf
Added gif creation to data cube visualization
Marshelix Aug 3, 2023
0ae255c
Fixed density
Marshelix Aug 3, 2023
c7e6ee9
Updated plotting code for data cube
Marshelix Aug 3, 2023
fdd5266
Add coupled requirement on velocity and density
Marshelix Aug 3, 2023
ba1dde2
Change velocity weight.
Aug 5, 2023
902e688
Added logging to model cube plot, tested parameters
Marshelix Aug 7, 2023
c9458d9
Density cube pixel-by-pixel evaluation.
Aug 8, 2023
fed259d
Print average mse of relative density and ground truth.
Aug 8, 2023
fa90c4a
Option to igore half of r. Normalise relative denstiy by mean. Use MA…
Aug 8, 2023
5cf6710
Added ability to plot vtk version of density cube, extraction of CME …
Marshelix Aug 8, 2023
5fc291f
Merge branch 'ground_density_cube_eval' into ground_params
Aug 8, 2023
3d3c92a
added mask_mode parameter for showing/filtering background points and…
hlamdouar Aug 9, 2023
56677f0
add batch loading of density cube
RobertJaro Aug 10, 2023
6526d60
Started probability module
Marshelix Aug 10, 2023
e712941
update loading vtk, add r2 filter
RobertJaro Aug 13, 2023
4be85fd
Initial estimate of hit probability
Marshelix Aug 14, 2023
b856056
Merge branch 'ground_params' of github.com:FrontierDevelopmentLab/202…
Marshelix Aug 14, 2023
0c43f55
Initial estimate of hit probability
Marshelix Aug 14, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion config/hyperparams_icarus.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ Model:
use_fine_model : True # If set, creates a fine model
d_filter_fine : 512 # Dimensions of linear layer filters of fine network
n_layers_fine : 8 # Number of layers in fine network bottleneck
d_output : 1 # electron density
d_output : 4 # electron density

Hierarchical sampling:
n_samples_hierarchical : 128 # Number of samples per ray
Expand Down
4 changes: 3 additions & 1 deletion config/psi_train.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,5 +6,7 @@ Training:

Lambda:
regularization: 1

continuity: 1.e-4
radial_regularization: 1.e-4
velocity_regularization: 1.e-4
Debug: False
21 changes: 14 additions & 7 deletions scripts/run_icarus.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,6 @@
# Training from scratch
# init workspace

# PSI
# convert data
python -m sunerf.prep.prep_psi_cor --psi_path "/mnt/ground-data/PSI/pb_raw/*.fits" --output_path "/mnt/prep-data/prep_PSI/pb_raw"
python -m sunerf.prep.prep_psi_cor --psi_path "/mnt/ground-data/PSI/b_raw/*.fits" --output_path "/mnt/prep-data/prep_PSI/b_raw"
# full training PSI
python -m sunerf.sunerf --wandb_name "psi" --data_path_pB "/mnt/ground-data/prep_PSI/pb_raw/*.fits" --data_path_tB "/mnt/ground-data/prep_PSI/b_raw/*.fits" --path_to_save "/mnt/training/PSI_v1" --train "config/train.yaml" --hyperparameters "config/hyperparams_icarus.yaml"

# HAO
# convert data
Expand Down Expand Up @@ -102,7 +96,8 @@ gsutil -m cp gs://fdl23_europe_helio_onground/ground-data/data_fits/dcmer_340W_
gsutil -m cp gs://fdl23_europe_helio_onground/ground-data/data_fits/dcmer_360W_bang_0000_tB/stepnum_005.fits /mnt/ground-data/data_fits/dcmer_360W_bang_0000_tB/stepnum_005.fits
gsutil -m cp gs://fdl23_europe_helio_onground/ground-data/data_fits/dcmer_360W_bang_0000_pB/stepnum_005.fits /mnt/ground-data/data_fits/dcmer_360W_bang_0000_pB/stepnum_005.fits


# Download for all of the PSI Data
gsutil -m cp -R gs://fdl23_europe_helio_onground/ground-data/PSI /mnt/ground-data/PSI/
################
# #
# Prep Data #
Expand All @@ -119,15 +114,27 @@ python -m sunerf.prep.prep_hao --resolution 512 --hao_path "/mnt/ground-data/dat
# prep_HAO_allview
python -m sunerf.prep.prep_hao --resolution 512 --hao_path "/mnt/ground-data/data_fits/**/*.fits" --output_path /mnt/prep-data/prep_HAO_allview --check_matching

# PSI
# convert data
python -m sunerf.prep.prep_psi_cor --psi_path "/mnt/ground-data/PSI/pb_raw/*.fits" --output_path "/mnt/prep-data/prep_PSI/pb_raw"
python -m sunerf.prep.prep_psi_cor --psi_path "/mnt/ground-data/PSI/b_raw/*.fits" --output_path "/mnt/prep-data/prep_PSI/b_raw"


######################
# #
# Running ICARUS #
# #
######################

#Prep_HAO_1view
python -m sunerf.sunerf --wandb_name "hao_pinn_1view" --data_path_pB "/mnt/prep-data/prep_HAO_1view/*pB*.fits" --data_path_tB "/mnt/prep-data/prep_HAO_1view/*tB*.fits" --path_to_save "/mnt/training/HAO_pinn_1view" --train "config/train.yaml" --hyperparameters "config/hyperparams_hao.yaml"

# Prep_HAO_2view
python -m sunerf.sunerf --wandb_name "hao_pinn_2view" --data_path_pB "/mnt/prep-data/prep_HAO_2view/*pB*.fits" --data_path_tB "/mnt/prep-data/prep_HAO_2view/*tB*.fits" --path_to_save "/mnt/training/HAO_pinn_2view" --train "config/train.yaml" --hyperparameters "config/hyperparams_hao.yaml"
# prep_HAO_2view_backgrounds
python -m sunerf.sunerf --wandb_name "hao_pinn_2view_background" --data_path_pB "/mnt/prep-data/prep_HAO_2view_background/*pB*.fits" --data_path_tB "/mnt/prep-data/prep_HAO_2view_background/*tB*.fits" --path_to_save "/mnt/training/HAO_pinn_2view_background" --train "config/train.yaml" --hyperparameters "config/hyperparams_hao.yaml"
# prep_HAO_allview
python -m sunerf.sunerf --wandb_name "hao_pinn_all" --data_path_pB "/mnt/prep-data/prep_HAO_allview/*pB*.fits" --data_path_tB "/mnt/prep-data/prep_HAO_allview/*tB*.fits" --path_to_save "/mnt/training/HAO_pinn_allview" --train "config/train.yaml" --hyperparameters "config/hyperparams_hao.yaml"

# full training PSI
python -m sunerf.sunerf --wandb_name "psi" --data_path_pB "/mnt/prep-data/prep_PSI/pb_raw/*.fits" --data_path_tB "/mnt/prep-data/prep_PSI/b_raw/*.fits" --path_to_save "/mnt/training/PSI_v1" --train "config/train.yaml" --hyperparameters "config/hyperparams_icarus.yaml"
Loading