Update the format of the trusted bidding signals

[MattMenke2]

This is a proposal to update the format of trusted bidding signals, to allow for different types of data to be provided. The immediate goal is to allow reprioritization/filtering information to be provided, along the lines discussed in issue #305. Maintains backwards compatibility with the old format (for now) so as not to break early experiments, but longer term, we'll want to remove the old format - could perhaps break compatibility once we start requiring JSON come from a validated trusted server.

Also adds the list of interest groups to the bidder JSON fetches (only), as those will also be needed for the filter logic, and may be generally useful.

This PS does not add anything specific to the filtering/reprioritization logic, or anything specific to one of the proposals in issue #305 - I think we'll need this new format for any filter API that we let use bidding signals. Seller signals, where we may or may not also implement filtering, already uses a format where new top-level dictionaries to be added, so does not need to have its format updated.

[jeffkaufman]

we'll want to remove the old format - could perhaps break compatibility once we start requiring JSON come from a validated trusted server.

As long as you gave ~2 milestones between adding the new format and removing the old one, and added a deprecation warning, I think the small number of groups actively experimenting would have time to switch?

Better to change fewer things at once, and switching to a validated trusted server is already a big change.

[stguav]

Couple of clarifications:

• what are the intended limits on how the trusted server can use the interestGroups data in preparing the response? My assumption is that the response should continue to only depend on the individual key and maybe its corresponding interest group, and the server should not be generating responses that depend on cross-IG membership information of the user. Is that correct?
• relatedly, are the interestGroups provided only the ones corresponding to the coalesced keys? Or would every request include all interestGroups with the trustedBiddingSignalsUrl regardless of how the keys are coalesced?

[MattMenke2]

we'll want to remove the old format - could perhaps break compatibility once we start requiring JSON come from a validated trusted server.

As long as you gave ~2 milestones between adding the new format and removing the old one, and added a deprecation warning, I think the small number of groups actively experimenting would have time to switch?

Better to change fewer things at once, and switching to a validated trusted server is already a big change.

We'll make sure folks have adequate time to switch - the trusted server will use an entirely different response packaging anyways (which I'm not familiar with), so only supporting the new format through that seems simplest, but I'll defer to the folks working on that side of things. It will be quite some time before we require trusted servers, regardless.

[MattMenke2]

Couple of clarifications:

• what are the intended limits on how the trusted server can use the interestGroups data in preparing the response? My assumption is that the response should continue to only depend on the individual key and maybe its corresponding interest group, and the server should not be generating responses that depend on cross-IG membership information of the user. Is that correct?

This PR is probably not the best place to ask this questions (since I hope to land this today, and I don't know the answers to it) - maybe file a new issue?

• relatedly, are the interestGroups provided only the ones corresponding to the coalesced keys? Or would every request include all interestGroups with the trustedBiddingSignalsUrl regardless of how the keys are coalesced?

Yes, it only includes the IGs that correspond to the coalesced keys. A lot of this information will be ironed out a bit more when we support trusted servers, where the request format will change a bit (We'll be exchanging essentially the same data, but in a slightly different format)

I've gone ahead and removed the "&interestGroups =..." part of the PR. I'll include it in the next PR where they're actually used in the response as well, keeping this one focused on the new response format, so we can land it promptly. Note that the new format is not yet supported by Chrome.

[JensenPaul]

Some nits, feel free to ignore or accept if you think they improve wording.

[maciejkowalczyk]

I've gone ahead and removed the "&interestGroups =..." part of the PR. I'll include it in the next PR where they're actually used in the response as well, keeping this one focused on the new response format, so we can land it promptly. Note that the new format is not yet supported by Chrome.

IIUC, with this change trusted server has no clue about the format version supported by the browser.
Perhaps the browser could send an expected/supported format version in the request?
Otherwise, we would not be able to implement the new protocol until its support lands in Beta (OT version).

[MattMenke2]

I've gone ahead and removed the "&interestGroups =..." part of the PR. I'll include it in the next PR where they're actually used in the response as well, keeping this one focused on the new response format, so we can land it promptly. Note that the new format is not yet supported by Chrome.

IIUC, with this change trusted server has no clue about the format version supported by the browser. Perhaps the browser could send an expected/supported format version in the request? Otherwise, we would not be able to implement the new protocol until its support lands in Beta (OT version).

This is true, but I'm hoping we can get away without adding more version negotation stuff. There's no real need to use the new format until we start using it, which will include adding the interestGroups request header, at least that's the current expectations, so I'm hoping we won't need to add more stuff around version negotiation - it'll just be another header to eventually deprecate and remove. I'm also thinking this will be the only time we need to update the version, since we can add query params and new response fields without breaking anything. We're also not planning on removing support for the old format until we remove support for the bring-your-own-server model, which is not happening anytime soon.

That having been said, I landed support for the new format earlier today, so I believe it will be in Canary 105.0.5190.0 (Possibly 105.0.5189.0), if you do want to experiment with it.