Add backend logic for bounding box plots

[julieg18]

main <- #5227 <- #5241 <- this <- #5305

This PR updates our comparison dvc output fixture with bounding box info and updates the backend accordingly. It also adds logic for a user toggling boxes on/off.

[julieg18]

I based the output off of iterative/dvclive#766 (comment) description, but this will most likely change and we will need to adjust accordingly.

[julieg18]

Updated to top, bottom, right, left based off the top-left corner after a discussion with Alex (see his comment):

[julieg18]

this.comparisonClassesSelected keeps track of what class labels (and by extension, the boxes) are on/off.

[sroy3]

What I meant by removing nested loops was not moving one inside of another function (I know realize we have 4 nested loops total), but can we change the code to drop some or at least make them more performant.

I see that in the fourth nested loop, we check for selected state. Maybe for a start, we could loop only through paths that are also in comparisonClassesSelected, since we only care about labels in comparisonClassesSelected[path].

There might be other optimizations to be made as well.

[julieg18]

What I meant by removing nested loops was not moving one inside of another function (I know realize we have 4 nested loops total), but can we change the code to drop some or at least make them more performant.

Thanks for the clarification!

I see that in the fourth nested loop, we check for selected state. Maybe for a start, we could loop only through paths that are also in comparisonClassesSelected, since we only care about labels in comparisonClassesSelected[path].

comparisonClassesSelected only holds classes that the user has toggled directly, it doesn't hold every visible class. Unless I'm missing something, we can't rely on it to detect which paths need to be looped over to collect classes.

The four loops are:

1. looping over each revision (up to 7)
2. looping over each path (could be a lot depending on how many plots the user has)
3. looping over each path img (will usually be just one, unless it's an image step plot)
4. looping over each class, gathering only the selected class boxes

I don't see a way to get rid of the first three loops, since for us to gather a plot's classes, we need to check for classes in each single image and collect the data. Atleast the third loop will likely just be a single item array.

We could get rid of the fourth loop by passing all image box data to the frontend and have ComparisonTableBoundingBoxImg filter out the boxes, but that means passing larger arrays that the component needs to loop over. Seems like we would want to limit the data passed to the frontend as much as possible but please correct me if I'm wrong.

Another alternative is having collectSelectedComparisonPlots gather class information along with the selected comparison plot data. I chose not to do this since that would make the function more complex, but if we think it's worth it for performance, I'm alright with doing that.

TL;DR

I'm not seeing a good way to get rid of any of our four loops. I'll review the loops and see what we can do to optimize them for now.

@sroy3, what do you think?

[julieg18]

Reviewed optimizing the loops! Instead of looping over every revision/path in comparisonData, we could start by looping over the already formed plots array. The plots already have a collected classDetails that tells you if a path has selected classes or not, lettings us skip early. Aka:

1. looping over each plots path (skip over if classDetails is empty)
2. looping over each revision
3. looping over each path img (will usually be just one, unless it's an image step plot)
4. looping over each class, gathering only the selected class boxes

Not seeing any other optimizations besides the two ideas I mentioned in my previous comment (have collectSelectedComparisonPlots collect box data, have ComparisonTableBoundingBoxImg do the box filtering).

[sroy3]

One thing I'm seeing is this:

if (selectedState === false) {
  continue
}

if (imgClasses.length === 0) {
  return
}

Why do we have to go through four loops to return nothing when we know it before hand? I would make this the first loop (the selected classes).

If there are no other possible optimizations, then I guess it's fine to keep as is.

[julieg18]

if (selectedState === false) {
continue
}
Why do we have to go through four loops to return nothing when we know it before hand? I would make this the first loop (the selected classes).

Aren't we already getting the selected classes in the first loop? In fact the line if (selectedState === false) {... no longer exists in the code. Did Github direct you the latest changes or is there something I'm missing here?

One thing I'm seeing is this:
if (imgClasses.length === 0) {
return
}

Good catch on if (imgClasses.length === 0) {...! I didn't spot the fact that we might be needlessy looping over selected classes if an img has no attached boxes to begin with. I'll update the code to check for img.boxes before running getSelectedImgComparisonPlotClasses.

[sroy3]

If this is empty, should this return {} as well? This would avoid having to loop through selectedRevisionIds for nothing.

[julieg18]

On further inspection, comparisonClassesSelected is only going to hold classes that the user has deselected/selected, it doesn't hold every visible class so we are unable to use it as a check for if there are selected classes or not.

[julieg18]

This variable probably needs to be renamed to avoid confusion. Maybe something like comparisonClassesSelectedState 🤔

[sroy3]

But then it's calling collectSelectedComparisonPlotClasses, which should return and empty object anyway, no? If we avoid calling it and the looping when we can, it's a good thing.

[julieg18]

No, collectSelectedComparisonPlotClasses could still return selected class boxes. Currently, we select all classes in a plot by default and comparisonClassesSelected only gets updated with a class (setting it as true or false) when a user clicks directly on a class button. Hope that makes sense :)

[sroy3]

But then it's calling collectSelectedComparisonPlotClasses, which should return and empty object anyway, no? If we avoid calling it and the looping when we can, it's a good thing.

[sroy3]

One thing I'm seeing is this:

if (selectedState === false) {
  continue
}

if (imgClasses.length === 0) {
  return
}

Why do we have to go through four loops to return nothing when we know it before hand? I would make this the first loop (the selected classes).

If there are no other possible optimizations, then I guess it's fine to keep as is.

[julieg18]

Going to go ahead and merge this and #5305 into our first PR since they've been sitting for a while and are approved.