Prefer safe logging preconditions in more places #6223

schlosna · 2022-09-09T03:45:06Z

General

Before this PR:
Nit comment on #6213 (comment)

we should clean up TableReference to use the com.palantir.logsafe.Preconditions instead of Guava, then we could get rid of this getTableRef(String) method and just inline the TableReference.createFromFullyQualifiedName(tableName)

After this PR:

==COMMIT_MSG==
Prefer safe logging preconditions in more places
==COMMIT_MSG==

Priority: P2

Concerns / possible downsides (what feedback would you like?):
This only addresses some atlasdb modules' migration of Preconditions from Guava to logsafe, many more (~53) remain.

Is documentation needed?:

Compatibility

Does this PR create any API breaks (e.g. at the Java or HTTP layers) - if so, do we have compatibility?:

Does this PR change the persisted format of any data - if so, do we have forward and backward compatibility?:

The code in this PR may be part of a blue-green deploy. Can upgrades from previous versions safely coexist? (Consider restarts of blue or green nodes.):

Does this PR rely on statements being true about other products at a deployment - if so, do we have correct product dependencies on these products (or other ways of verifying that these statements are true)?:

Does this PR need a schema migration?

Testing and Correctness

What, if any, assumptions are made about the current state of the world? If they change over time, how will we find out?:

What was existing testing like? What have you done to improve it?:

If this PR contains complex concurrent or asynchronous code, is it correct? The onus is on the PR writer to demonstrate this.:

If this PR involves acquiring locks or other shared resources, how do we ensure that these are always released?:

Execution

How would I tell this PR works in production? (Metrics, logs, etc.):

Has the safety of all log arguments been decided correctly?:

Will this change significantly affect our spending on metrics or logs?:

How would I tell that this PR does not work in production? (monitors, etc.):

If this PR does not work as expected, how do I fix that state? Would rollback be straightforward?:

If the above plan is more complex than “recall and rollback”, please tag the support PoC here (if it is the end of the week, tag both the current and next PoC):

Scale

Would this PR be expected to pose a risk at scale? Think of the shopping product at our largest stack.:

Would this PR be expected to perform a large number of database calls, and/or expensive database calls (e.g., row range scans, concurrent CAS)?:

Would this PR ever, with time and scale, become the wrong thing to do - and if so, how would we know that we need to do something differently?:

Development Process

Where should we start reviewing?:

If this PR is in excess of 500 lines excluding versions lock-files, why does it not make sense to split it?:

Please tag any other people who should be aware of this PR:
@jeremyk-91
@sverma30
@raiju

changelog-app · 2022-09-09T03:45:10Z

Generate changelog in `changelog/@unreleased`

Type

Description

Prefer safe logging preconditions in more places

Check the box to generate changelog(s)

Generate changelog entry

schlosna · 2022-09-11T14:59:38Z

...-client/src/main/java/com/palantir/atlasdb/table/description/render/StreamStoreRenderer.java

@@ -305,7 +306,7 @@ private void touchMetadataWhileStoringForConflicts() {
                    line("StreamMetadata metadata ="
                            + " metaTable.getMetadatas(ImmutableSet.of(row)).values().iterator().next();");
                    line("Preconditions.checkState(metadata.getStatus() == Status.STORING, \"This stream is being"
-                            + " cleaned up while storing blocks: %s\", id);");
+                            + " cleaned up while storing blocks\", SafeArg.of(\"id\", id));");


Generally these IDs are just opaque identifiers, so I'd lean toward safe, but question for reviewers -- should we consider these safe or unsafe?

Yeah, this is a timestamp (so it should be safe!)

schlosna · 2022-09-11T15:00:46Z

...-client/src/main/java/com/palantir/atlasdb/table/description/render/StreamStoreRenderer.java

+                        line("        SafeArg.of(\"streamId\", e.getKey().getId()),");
+                        line("        SafeArg.of(\"status\", metadata.getStatus()));");


similar question as above around safe/unsafe for streamId which should be an opaque identifier, and status is enum so should be safe

streamId is a timestamp, so safe.

schlosna · 2022-09-11T15:01:29Z

atlasdb-commons/src/main/java/com/palantir/util/OptionalResolver.java

+                    "Contradictory values found, expected a single common value", UnsafeArg.of("values", values));
+        } else {
+            throw new SafeIllegalArgumentException("All Optionals provided were empty, couldn't determine a value.");
+        }


inlined a bit here to avoid arg allocations in happy paths

schlosna · 2022-09-11T15:02:06Z

atlasdb-commons/src/test/java/com/palantir/common/concurrent/CoalescingSupplierTest.java

+            } else {
+                throw new SafeIllegalStateException(
+                        "current > last", SafeArg.of("current", current), SafeArg.of("last", last));
+            }


similar inlining to avoid arg allocations on happy paths

schlosna · 2022-09-11T15:10:06Z

.gitattributes

@@ -0,0 +1 @@
+**/generated/** linguist-generated


Mark generated files as linguist-generated to hide GitHub PR diff
display by default for generated schema file changes.

See GitHub Docs:
https://docs.github.com/en/repositories/working-with-files/managing-files/customizing-how-changed-files-appear-on-github
https://github.com/github/linguist/blob/master/docs/overrides.md#generated-code

jeremyk-91

This all looks good - maybe for some of the table stuff we can use LoggingArgs because in some cases we know the table names are safe.

jeremyk-91 · 2022-09-21T09:25:56Z

atlasdb-cli/src/test/java/com/palantir/atlasdb/cli/command/TestSingleBackendCommand.java

-                        table.getQualifiedName());
+                        "kvs contains tables, but not table",
+                        UnsafeArg.of("tables", services.getKeyValueService().getAllTableNames()),
+                        UnsafeArg.of("table", table.getQualifiedName()));


Probably worth using LoggingArgs for table-related things: there may be some metadata we can use about which tables are safe or unsafe (e.g. all atlas defined internal tables are safe). In particular LoggingArgs.tableRef(...) or
SafeArg.of(name, LoggingArgs.safeTablesOrPlaceholder(...) might be useful

jeremyk-91 · 2022-09-21T09:26:18Z

...-client/src/main/java/com/palantir/atlasdb/table/description/render/StreamStoreRenderer.java

+                        line("        SafeArg.of(\"streamId\", e.getKey().getId()),");
+                        line("        SafeArg.of(\"status\", metadata.getStatus()));");


streamId is a timestamp, so safe.

jeremyk-91 · 2022-09-21T09:30:26Z

...sdb-dbkvs-hikari/src/main/java/com/palantir/nexus/db/pool/config/OracleConnectionConfig.java

-                    "truststore file not found at %s",
-                    getTruststorePath().get());
+                    "truststore file not found",
+                    UnsafeArg.of("truststorePath", getTruststorePath().get()));


On one hand this should normally come from config and so should normally be safe. There isn't necessarily a guarantee in the code that it does, though, and so I think this is probably a safer decision.

jeremyk-91 · 2022-09-21T09:31:46Z

...-client/src/main/java/com/palantir/atlasdb/table/description/render/StreamStoreRenderer.java

@@ -305,7 +306,7 @@ private void touchMetadataWhileStoringForConflicts() {
                    line("StreamMetadata metadata ="
                            + " metaTable.getMetadatas(ImmutableSet.of(row)).values().iterator().next();");
                    line("Preconditions.checkState(metadata.getStatus() == Status.STORING, \"This stream is being"
-                            + " cleaned up while storing blocks: %s\", id);");
+                            + " cleaned up while storing blocks\", SafeArg.of(\"id\", id));");


Yeah, this is a timestamp (so it should be safe!)

Mark generated files as linguist-generated to hide GitHub PR diff display by default for generated schema file changes. See GitHub Docs: https://docs.github.com/en/repositories/working-with-files/managing-files/customizing-how-changed-files-appear-on-github https://github.com/github/linguist/blob/master/docs/overrides.md#generated-code

schlosna mentioned this pull request Sep 9, 2022

Remove the conjure-java-jersey-server dependency #6213

Merged

schlosna commented Sep 11, 2022

View reviewed changes

jeremyk-91 approved these changes Sep 21, 2022

View reviewed changes

schlosna added the merge when ready label Nov 11, 2022

schlosna marked this pull request as ready for review November 11, 2022 22:09

schlosna and others added 10 commits November 28, 2022 15:02

Tests use logsafe Preconditions

aed214b

atlasdb-api uses logsafe Preconditions

1fbecc6

atlasdb-commons uses logsafe Preconditions

680a177

StreamStoreRenderer uses logsafe Preconditions

e35bd15

./gradlew generateSchemas

4664b85

atlasdb-dbkvs safe-logging Preconditions

2a922e6

Add generated changelog entries

47c09cc

StreamStoreRenderer format code

5764316

./gradlew generateSchemas

753983f

schlosna force-pushed the ds/PreferSafeLoggingPreconditions branch from b63e31d to 2810c38 Compare November 28, 2022 20:11

bulldozer-bot bot merged commit f9fb64d into develop Nov 28, 2022

bulldozer-bot bot deleted the ds/PreferSafeLoggingPreconditions branch November 28, 2022 20:31

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Prefer safe logging preconditions in more places #6223

Prefer safe logging preconditions in more places #6223

schlosna commented Sep 9, 2022

changelog-app bot commented Sep 9, 2022 •

edited by schlosna

Loading

schlosna Sep 11, 2022

jeremyk-91 Sep 21, 2022

schlosna Sep 11, 2022

jeremyk-91 Sep 21, 2022

schlosna Sep 11, 2022

schlosna Sep 11, 2022

schlosna Sep 11, 2022

jeremyk-91 left a comment

jeremyk-91 Sep 21, 2022

jeremyk-91 Sep 21, 2022

jeremyk-91 Sep 21, 2022

jeremyk-91 Sep 21, 2022

		line(" SafeArg.of(\"streamId\", e.getKey().getId()),");
		line(" SafeArg.of(\"status\", metadata.getStatus()));");

Prefer safe logging preconditions in more places #6223

Prefer safe logging preconditions in more places #6223

Conversation

schlosna commented Sep 9, 2022

General

Compatibility

Testing and Correctness

Execution

Scale

Development Process

changelog-app bot commented Sep 9, 2022 • edited by schlosna Loading

Generate changelog in changelog/@unreleased

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jeremyk-91 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

changelog-app bot commented Sep 9, 2022 •

edited by schlosna

Loading

Generate changelog in `changelog/@unreleased`