[EdgeDB] - Add trigger to appropriately create financial/narrative report upon project creation

[bryanjnelson]

Monday Story

Monday Task

Description

This adds a createPeriodicReportsOnInsert trigger to create the appropriate number of both narrative and financial reports when a project is created, based upon the financialReportPeriod in the project.

Open items for discussion/modification:

• Should the create_periodic_report_ranges function be made more generic to handle other use cases? (Probably not.)
• Is the create_periodic_report_ranges function located in the best place? It could be moved to the common.esdl, but if this is the only place it will be used then no. It could also potentially be moved to the periodic-report.esdl?
• It's possible the EdgeQL in create_periodic_report_ranges could be written a bit more cleanly
• It's currently problematic that the mouStart, mouEnd, and financialReportPeriod fields are not required on a Project; this will obviously prevent any reports from being generated correctly. We'll need to figure out the best way to handle that. Make them required? Update the trigger to also run on update when all 3 are finally populated?
• Handle TODO in the FinancialReport insert; I'm unclear what that project level field is used for

Ready for review checklist

Use [N/A] if the item is not applicable to this PR or remove the item

• Change the task url above to the actual Monday task
• Add reviewers to this PR

[CarsonF]

Wow love the initiative!

For this to be fully out of app code, we also need to handle:

• Date range expanding (on update), which would add new periods.
  Jan-March -> Jan-July
• Date range shrinking (on update), which would delete empty periods
  Jan-May -> Jan-March
  But we only want to delete reports that don't have a file uploaded or a progress report started
• On financial reporting frequency change create new and delete old empty periods.
  Monthly (Jan, Feb, March) -> Quarterly (Jan-March)

Honestly "on insert" will never be hit by users since they don't define the date ranges when they create the project.
May be useful for migration though?

Should the create_periodic_report_ranges function be made more generic to handle other use cases? (Probably not.)

Probably not

Is the create_periodic_report_ranges function located in the best place? It could be moved to the common.esdl, but if this is the only place it will be used then no. It could also potentially be moved to the periodic-report.esdl?

Probably their own files for each concrete report. They all have distinct use-cases / logic.

It's currently problematic that the mouStart, mouEnd, and financialReportPeriod fields are not required on a Project; this will obviously prevent any reports from being generated correctly. We'll need to figure out the best way to handle that. Make them required? Update the trigger to also run on update when all 3 are finally populated?

That's what we do in app code. If all 3 are given, synchronize.

[CarsonF]

This is still needed. The typecasting workaround was fixed though

  function date_range_get_upper(period: range<cal::local_date>) -> cal::local_date
    using (assert_exists(range_get_upper(period)) - <cal::date_duration>"1 day");

Ranges are half closed [start, end) meaning the endpoint is not considered to be included.
This matters when we start doing math, like does x date fall within this range?
For example this work week. We see it as

[2024-09-02, 2024-09-06]

but SQL/EdgeQL stores that as

[2024-09-02, 2024-09-06)

meaning the 6th is NOT included in that range.

That's why we add one day

[2024-09-02, 2024-09-07) == [2024-09-02, 2024-09-06]

And that's why this function exists, because when we display the "end date" we want it to be the inclusive endpoint, not the half closed one.

[bryanjnelson]

I think I understand what you're saying conceptually, but something still isn't adding up for me in practice.

When I run the create_periodic_report_ranges in the context of the trigger I created I receive the expected results. For example, if I call create_periodic_report_ranges alone in the editor like this:

create_periodic_report_ranges('2023-01-01', '2023-12-30', 3)

I receive this output:

[
  {"lower": "2023-01-01T00:00:00", "upper": "2023-03-31T00:00:00", "inc_lower": true, "inc_upper": false},
  {"lower": "2023-04-01T00:00:00", "upper": "2023-06-30T00:00:00", "inc_lower": true, "inc_upper": false},
  {"lower": "2023-07-01T00:00:00", "upper": "2023-09-30T00:00:00", "inc_lower": true, "inc_upper": false},
  {"lower": "2023-10-01T00:00:00", "upper": "2023-12-31T00:00:00", "inc_lower": true, "inc_upper": false},
  ...
]

which seems accurate and what we'd want.

Now if I actually run the trigger and then look at the data on the start and end fields on the Financial Report in the data explorer I see this:

[
  {"lower": "2023-01-01T00:00:00", "upper": "2023-03-30T00:00:00", "inc_lower": true, "inc_upper": false},
  {"lower": "2023-04-01T00:00:00", "upper": "2023-06-29T00:00:00", "inc_lower": true, "inc_upper": false},
  {"lower": "2023-07-01T00:00:00", "upper": "2023-09-29T00:00:00", "inc_lower": true, "inc_upper": false},
  {"lower": "2023-10-01T00:00:00", "upper": "2023-12-30T00:00:00", "inc_lower": true, "inc_upper": false},
  ...
]

The end dates seem to be one less than what I would expect to see. I understand that they are calculated values applying the custom date_range_get_upper function and thus subtracting a day, but is that what we want?

I do see that the period on the Financial Report is correct, however.

...

I guess the only thing I can think of is that I'm incorrect in my assessment that we really want a financial reporting period to look like this: 2023-01-01 -- 2023-03-31. Perhaps conceptually we want to think of the period as 2023-01-01 -- 2023-04-01? If that's the case then I'm just creating the periods in my function with an end date 1 less than we desire?

[CarsonF]

This is wild. Just noting that I have not vetted this for accuracy.

[bryanjnelson]

@CarsonF - After the trigger code settles down I think there's some opportunity for some common methods like I started here.

[CarsonF]

I don't think functions can mutate data right now. I don't really understand the limitation.

[CarsonF]

I believe you want ?!= on these 3 comparisons. Since they are all optional fields. We want null on either side to count as a valid value to compare against, instead of skipping the comparison all together when null is encountered. It's a semi well known footgun that's complained about. Docs have notes on these ?-prefixed operators.

[bryanjnelson]

@CarsonF - I'm getting this error here.

I'm trying to avoid ProgressReports here since that's a different flow. Any ideas on the best way to handle this?

[bryanjnelson]

@CarsonF - I'm wondering if this is the place where we can also handle the case in which all 3 of these values exist for the first time? We could check if they've each changed, but also that none of them are null?

On a related note, can any of these values go back to null once set?