Feature EV controlled charging custom constraints

[adrienmellot]

Fixes #350.
Fixes #81.

Within the realm of full electrification of road transport, this PR allows the controlled charging share of this electricity demand to be constrained.

Possibly, we want to have the run.py file to be integrated in a better way, and also in workflow tests.

Checklist

Any checks which are not relevant to the PR can be pre-checked by the PR creator. All others should be checked by the reviewer. You can add extra checklist items here if required by the PR.

• CHANGELOG updated
• Minimal workflow tests pass
• Tests added to cover contribution
• Documentation updated
• Configuration schema updated

[brynpickering]

Thanks for this @adrienmellot, shows how complex our EV modelling is. Too complex...?

Anyway, some reasonably major comments about reorganising file structures:

1. run.py could be restructured into a snakemake rule script that calls constraint adding scripts on-the-fly
2. the controlled EV annual demand and installed capacities rule should probably be split in two since they have some mutually exclusive inputs/params. They can use the same script (since they share the regionalisation step) but some parameter / wildcard is used in the script to decide whether df_demand or df_charging_potentials is generated.

[brynpickering]

I'd suggest having a run.py in scripts and then a add_constraints.py in the scripts/transport/ directory. The run.py has most of the content of this file except the constraints. It then lods constraints as modules and loads them into the model:

import importlib.util


model = calliope.read_netcdf(path_to_temp_model)
model.run(build_only=True)

for file in constraint_files:
    spec=importlib.util.spec_from_file_location("constraints", file)

    # creates a new module based on spec
    constraint_adder = importlib.util.module_from_spec(spec)

    # executes the module in its own namespace
    # when a module is imported or reloaded.
    spec.loader.exec_module(constraint_adder)

    constraint_adder.add_eurocalliope_constraints(model)

new_model = model.backend.rerun()
new_model.to_netcdf(path_to_output)

Then you can have the run step be a snakemake rule with the reference to the constraint scripts in the inputs.

[brynpickering]

This is untested by me, loading files as modules might need some trial and error work

[adrienmellot]

@brynpickering I've addressed the first part of the comments regarding style, not yet the part concerning the run.py and the file structure for the custom constraints.

Regarding your comment about splitting the controlled EV annual demand and installed capacities rule into two rules, I've discussed it with Tim and it makes life harder to split it. The reason for that is that the rule outputs one .csv file only which is later read to generate the .yaml for the model.

Splitting the rule into two would either require generating two .csv files, which makes it harder for the .yaml file to be generated afterwards (or two .yaml files, which we don't really want), or to have a rule order where one rule first generates a .csv and the second rule appends to this. All in all, a little more complex than having just one rule?

[brynpickering]

OK, it's because the template filler can only handle one location-specific CSV. This probably needs fixing as it leads to quite messy implementations like this. Not something to solve in this PR!

[adrienmellot]

For now custom constraints are not implemented (#385). I've thus removed the run.py file that there was to avoid any confusion. We discussed that the data processing could still be let through even though for now it is not immediately useful. I haven't updated the changelog and documentation for this PR as I don't think that it would not make much sense for now, it should probably be done once #385 is solved!

[brynpickering]

Final set of suggestions, otherwise OK to go ahead!

[brynpickering]

Suggested change

	country_codes=([
	pycountry.countries.lookup(c).alpha_3 for c in snakemake.params.countries
	]),
	country_codes=utils.convert_valid_countries(snakemake.params.countries).values()

[brynpickering]

Suggested change

	import pycountry
	from eurocalliopelib import utils

[brynpickering]

Give unit here rather than as a comment in the config

[brynpickering]

Actually, can't approve - it needs a changelog entry ;)