Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add functionality to check domains #61

Open
gregbunce opened this issue Nov 12, 2020 · 7 comments
Open

add functionality to check domains #61

gregbunce opened this issue Nov 12, 2020 · 7 comments

Comments

@gregbunce
Copy link
Contributor

currently, we are moving away from coded value domains that do not match the domain description. this check would flag domains that are not in compliance. see this doc for more info

@steveoh
Copy link
Member

steveoh commented Nov 12, 2020

would a possible fix be a new field with the value data calc'd?

@stdavis
Copy link
Member

stdavis commented Nov 12, 2020

I think that the fix could be to re-calc the existing codes with the descriptions in-place and update the domain codes to match the descriptions in the GDB domain.

@stdavis
Copy link
Member

stdavis commented Nov 12, 2020

Also, this is a duplicate of #56. I'll close that one since this issue has more discussion.

@steveoh
Copy link
Member

steveoh commented Nov 12, 2020

I think that the fix could be to re-calc the existing codes with the descriptions in-place and update the domain codes to match the descriptions in the GDB domain.

This would be a breaking change though right not an enhancement?

@ZachBeck
Copy link
Member

wouldn't you run into field length problems if you re-calc the code with the description?

@stdavis
Copy link
Member

stdavis commented Nov 12, 2020

Good points. Didn't think about the field length issues, but that could be worked around in code (create a temp new field).

As for breaking changes, I'm not sure that I see that. The other fixes in sweeper edit data as well. Agreed that it is more than just removing dups or empty geometry. I guess that if you just calc a new field, I don't see how that is helpful. Sweeper would still find the same errors the next time you run it.

@steveoh
Copy link
Member

steveoh commented Nov 12, 2020

This proposal is based on the situation doc greg linked to above. I will highlight some important sections.

The effects of abruptly changing these existing non-matching domains in the database is not fully understood at this time, but we do know that it will affect existing apps, scripts, workflows, data models, schemas, and field types.

I'll add that it might affect pro project symbology.

[For the affected datasets] create new corresponding domains in which the code matches the description. These can be used for testing and then migration moving forward.

With that in mind, sweeper would require two fixing scenarios; One where we keep the original field and domain and add a new field and correct domain and one where we replace it. The sweeping should be able to handle both cases.

sweep

  1. find all fields using coded value domains
  2. if any are found
    1. check for the existence of a field with the same name suffixed with some text that we can come up with as a group
    2. if the field doesn't exist check that the codes match the value
  3. report errors/warnings

fix

migratory fix (existing data)

  1. create new coded value domain with suffix where code equals the value
  2. calc temp field with values equaling the description of the domain being used
  3. apply new domain to field

permanent fix (new data)

  1. calc temp field with values equaling the description of the domain being used
  2. create new coded value domain where codes equal the value
  3. delete original coded value field
  4. delete original coded value domain
  5. apply new domain to field
  6. rename temp field to original

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants