Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Google #7241

Open
wants to merge 1,675 commits into
base: master
Choose a base branch
from
Open

Google #7241

wants to merge 1,675 commits into from

Conversation

Reaper333gang
Copy link

The file dependency data is used to check if a cached analysis object
is fresh.

In this initial phase, we only write the file dependency data without
having clients consume it. Clients skip the file dependency data reference
present in the cached value.

Now that the logic is a bit more complex, splits out the
SelectedEntrySerializer to serialize specific selected values from the
FrontierSerializer, in addition to the file dependency data.

This change adds a couple of test helpers.

  • LongVersionGetterTestInjection - to inject LongVersionGetter in tests
  • FakeInvalidationDataHelper - to add fake invalidation data to test generated
    cache values.

PiperOrigin-RevId: 718118838

Checklist

  • I have filed an issue about this change and discussed potential changes with the maintainers.
  • I have received the approval from the maintainers to make this change.
  • This is not a stylistic, refactoring, or cleanup change.

Please note that the maintainers will not be reviewing this change until all checkboxes are ticked. See
the Contributions section in the README for more
details.

Discussion thread for this change

Issue number: <please reference the issue number or url here>

Description of this change

Googler and others added 30 commits February 29, 2024 18:11
…ices/grpc:state_provider

PiperOrigin-RevId: 611659485
... and expose new dependencies

PiperOrigin-RevId: 611825305
PiperOrigin-RevId: 611838288
PiperOrigin-RevId: 612007503
and normalize path handling and testing at the same time.

PiperOrigin-RevId: 612010943
PiperOrigin-RevId: 612017172
to allow upcoming `CitcOperations` implementation for `intellij-ext`

PiperOrigin-RevId: 612042193
PiperOrigin-RevId: 612093401
PiperOrigin-RevId: 612474984
Implementation to follow in future CLs.

PiperOrigin-RevId: 612751070
…ent in Android Studio.

Also ensure that bazel with clion remains usable for all C++ development.

PiperOrigin-RevId: 612765976
There is nothing query sync specific in here, and putting it in the common package allows to to share implementation details with other classes in there in future.

PiperOrigin-RevId: 612840186
The purpose of this is to ensure that protos that contains duplicated strings do not result in duplicated strings being retained in memory. Passing all proto messages through the interned immediately after deserialization will result in all strings with them being interned, allowing all duplicated strings to be garbage collected.

As an optimization, we also ensure that proto messages that do not contain any strings are not copied.

PiperOrigin-RevId: 612846381
This contains basic information about a dependency build and will be used to track and identify builds and the depdendency artifacts created by them.

PiperOrigin-RevId: 612848102
When parsing the query output, or de=serializing a query summary proto, pass all protos through a proto string interner. This elimiates duplicate string instances in memory and reduces memory usage.

PiperOrigin-RevId: 612849276
If true, returns a list of paths to replace within the `ExecRequest` to make the resulting paths portable.

Note that this is not published to the BES, just the `ExecRequest` returned over grpc.

PiperOrigin-RevId: 612928660
PiperOrigin-RevId: 613143912
to make them analyzable.

PiperOrigin-RevId: 613172763
Requiring an instance to be created is not necessary and makes using it a little more awkward in some cases. We never expect to use it with an interner other than that provided by `StringInterner` anyway.

PiperOrigin-RevId: 613173200
…ate.

Intern the proto strings when reading protos after a dependencies build, and again when restoring the same data from disk. In `JavaArtifactInfo`, also intern the paths we derive from the proto contents. It seems to be worth interning the paths themselves, since they memoize the `toString()` representation internally which we can't memoize directly.

Based on a little experimentation, this reduces the size of the artifact tracker state to about 5-10% of what it was before.

We add a path interner to the `StringInterner` class, and also rename it to `Interners` since it now deals with more than just strings.

PiperOrigin-RevId: 613190920
PiperOrigin-RevId: 613219486
PiperOrigin-RevId: 613252380
…heir WORKSPACE.

PiperOrigin-RevId: 613276608
Googler and others added 18 commits November 20, 2024 07:47
Bazel exposes BUILD files (and protos) that use protobuf in @bazel_tools. Until this is used in WORKSPACE mode, the name needs to stay com_google_protobuf.

Replace @protobuf// with @com_google_protobuf. This is the trivial change.

Fix WORKSPACE.suffixes. They were generated with @protobuf, instead of @com_google_protobuf.

Fix autoloads to use @com_google_protobuf. The WORKSPACE.suffix files were obscuring the problem here.

Upgrade rules_cc to 0.0.13, rules_java to 8.3.1, those are the first using again @com_google_protobuf.

Upgrade protobuf to 29.0-rc3. Matches MODULE.bazel and has rules implementation.

Fixes: bazelbuild/bazel#24401
PiperOrigin-RevId: 698388130
PiperOrigin-RevId: 700289370
This is always under OUTPUT_BASE, so redundant.

PiperOrigin-RevId: 700701273
…attribute inheritance

Symbolic macros have a mandatory visibility attribute - we need to document it.

To support attribute inheritance in documentation generation, it suffices to
extract non-Starlark-defined attributes in macros, and expose the fact that
they are non-Starlark-defined - so that a documentation renderer can provide
a link to bazel.build common attributes for the natively defined common
attributes that don't have a doc string.

Take the opportunity to refactor addDocumentableAttributes() into a better
shape (taking an explicit argument for the implicitly-added attributes), and
fix the long-standing bug that Stardoc has been inserting a non-existent
"name" attribute for aspects (which was partly obscured by the old
addDocumentableAttributes() implementation).

RELNOTES: Fix starlark_doc_extract proto output for symbolic macro visibility,
attribute inheritance, and rule finalizers; and remove non-existent "name"
attribute from starlark_doc_extract output for aspects.
PiperOrigin-RevId: 700811908
[
PiperOrigin-RevId: 701004715
 - Replaces `@protobuf` with `@com_google_protobuf`[^1]
 - Updates protobuf version to the latest release
 - Updates versions & moves declaration of `rules_python` and `rules_proto` before calling any setup macros to ensure we're on the right version.

Work towards bazelbuild/bazel#23619

PiperOrigin-RevId: 701894382
Fixes #24611.

RELNOTES: None.
PiperOrigin-RevId: 704618999
* Update testing plaforms

gcc compiler on ubuntu1804 is too old for Bazel@HEAD

https://buildkite.com/bazel/bazel-at-head-plus-downstream/builds/4152#019231ab-a1e4-46bd-916c-7be8b8028756

* More updates

(cherry picked from commit 36fc091)

Co-authored-by: Yun Peng <[email protected]>
PiperOrigin-RevId: 707279125
…ING_DIRECTORY`

In more details:

* A new `PathToReplace#Type` enum value (`TEMP_LOGGING_DIRECTORY`) is added.
* `localTempLoggingDirPathStr` can be injected into `InstrumentationOutputFactory`.
* For local output, blaze crashes if relative to type is `TEMP_LOGGING_DIRECTORY` but `localTempLoggingDirPathStr` is not provided.
* `createInstrumentationOutput()` might optionally take in `createParent` flag, which defaults to `false`.

PiperOrigin-RevId: 710150887
This never wound up being used (including never populated in bazel) and adds
some complexity we can do without.

PiperOrigin-RevId: 711752166
PiperOrigin-RevId: 712443020
[
PiperOrigin-RevId: 716297105
The file dependency data is used to check if a cached analysis object
is fresh.

In this initial phase, we only write the file dependency data without
having clients consume it. Clients skip the file dependency data reference
present in the cached value.

Now that the logic is a bit more complex, splits out the
SelectedEntrySerializer to serialize specific selected values from the
FrontierSerializer, in addition to the file dependency data.

This change adds a couple of test helpers.
* LongVersionGetterTestInjection - to inject LongVersionGetter in tests
* FakeInvalidationDataHelper - to add fake invalidation data to test generated
  cache values.

PiperOrigin-RevId: 718118838
@github-actions github-actions bot added product: CLion CLion plugin product: IntelliJ IntelliJ plugin product: GoLand GoLand plugin awaiting-review Awaiting review from Bazel team on PRs labels Jan 22, 2025
@sellophane
Copy link
Collaborator

Hi, are you sure you're intending to merge the correct branch here?

@sgowroji sgowroji added awaiting-user-response Awaiting response from author on PRs and removed awaiting-review Awaiting review from Bazel team on PRs labels Jan 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
awaiting-user-response Awaiting response from author on PRs product: CLion CLion plugin product: GoLand GoLand plugin product: IntelliJ IntelliJ plugin
Projects
Status: Untriaged
Development

Successfully merging this pull request may close these issues.