Missing value location error from projectField

[lee1043]

Error occurs from projectField when data was opened using xr.open_mfdataset and solver was using weights. I have doubled check there was no error with the input data, which has no missing value at all.

import xarray as xr
from eofs.xarray import Eof

demo_file = "ts_Amon_ACCESS1-0_historical_r1i1p1_185001-200512.nc"
data_var = "ts"

#
# test 1: open dataset using 'open_dataset'
#
ds = xr.open_dataset(demo_file)
ds = ds.isel(time=slice(0,20))
da = ds[data_var]
wtarray = xr.ones_like(da)

# test 1a: working okay
solver = Eof(da)
pseudo_pcs = solver.projectField(da)

# test 1b: weights used for solver, working okay
solver = Eof(da, weights=wtarray)
pseudo_pcs = solver.projectField(da)

#
# test 2: open dataset using 'open_mfdataset'
#
ds = xr.open_mfdataset(demo_file)
ds = ds.isel(time=slice(0,20))
da = ds[data_var]
wtarray = xr.ones_like(da)

# test 2a: working okay
solver = Eof(da)
pseudo_pcs = solver.projectField(da)

# test 2b: error
solver = Eof(da, weights=wtarray)
pseudo_pcs = solver.projectField(da)

Error message:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[10], line 2
      1 solver = Eof(da, weights=wtarray)
----> 2 pseudo_pcs = solver.projectField(da)

File ~/mambaforge/envs/pmp_devel_20230223/lib/python3.9/site-packages/eofs/xarray.py:618, in Eof.projectField(self, array, neofs, eofscaling, weighted)
    616 else:
    617     has_time = False
--> 618 pcs = self._solver.projectField(array.values,
    619                                 neofs=neofs,
    620                                 eofscaling=eofscaling,
    621                                 weighted=weighted)
    622 # Create the PCs DataArray.
    623 if pcs.ndim == 2:

File ~/mambaforge/envs/pmp_devel_20230223/lib/python3.9/site-packages/eofs/standard.py:752, in Eof.projectField(self, field, neofs, eofscaling, weighted)
    748 eofNonMissingIndex = np.where(
    749     np.logical_not(np.isnan(self._flatE[0])))[0]
    750 if eofNonMissingIndex.shape != nonMissingIndex.shape or \
    751         (eofNonMissingIndex != nonMissingIndex).any():
--> 752     raise ValueError('field and EOFs have different '
    753                      'missing value locations')
    754 eofs_flat = self._flatE[slicer, eofNonMissingIndex]
    755 if eofscaling == 1:

ValueError: field and EOFs have different missing value locations

I suspect this might be somewhat related to #122. Does anyone have any idea?

[lee1043]

Thank you, @ajdawson, for providing and sharing this amazing package. It has been very helpful for my project, developing a tool to collectively evaluate climate models, named PCMDI Metrics Package. In my recent work to convert the code that used to be cdms based to use xarray, because the lifespan of the cdms is sunsetting, I have encountered the above issue. I wonder if you have any clue on this.

[lee1043]

I just found the the fix suggested by @lfery resolves my issue, and it passes all 4 tests above. I will write open a pull request for that. -- Thank you, @lfery!