Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rework auto begin/commit/rollback transaction during query lifetime #4674

Merged
merged 2 commits into from
Jan 2, 2025

Conversation

ray6080
Copy link
Contributor

@ray6080 ray6080 commented Jan 2, 2025

Description

This PR fixes two problems related to the usage of transactions:

  1. Parser relies on transaction to resolve data type for default null value, which should be done in the binder phase.
  2. Parser and prepare unnecessarily starts and commits transactions, leading to multiple transactions in a single statement.

For the first problem, the PR moves resolving data type of default null from Parser to Binder, thus, Parser doesn't rely on transactions.
For the second problem, the PR unifies how transactions should auto begin and commit/rollback in TransactionHelper::runFuncInTransaction(), and avoids unnecessary begin/commits of transactions.

This comment was marked as outdated.

@ray6080 ray6080 requested review from andyfengHKU and removed request for benjaminwinger January 2, 2025 08:19

This comment was marked as outdated.

src/processor/operator/simple/export_db.cpp Outdated Show resolved Hide resolved
std::vector<PropertyDefinition> Binder::bindPropertyDefinitions(
const std::vector<ParsedPropertyDefinition>& parsedDefinitions, const std::string& tableName) {
std::vector<PropertyDefinition> definitions;
for (auto& parsedDefinition : parsedDefinitions) {
auto type = LogicalType::convertFromString(parsedDefinition.getType(), clientContext);
// For default null value, we may need to resolve its data type, as its type may not be
// resolved during parsing and thus may be ANY.
tryResolvingDataTypeForDefaultNull(parsedDefinition, type);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cannot this be handled with implicitCast in l79? I don't think this is a good design.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually don't know. Let me give a try, if implicitCast can handle this, it would be great.

Copy link
Contributor Author

@ray6080 ray6080 Jan 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I cannot directly use implicitCast here because the boundExpr—after binding and implicit casting—is not actually utilized in PropertyDefinition. Instead, PropertyDefinition continues to hold the parsed expression, which is not cast.

I believe the current approach of storing ParsedExpression in PropertyDefinition is not well designed. This method essentially discards the results of binding and casting (which are only used for validation) and re-binds the default expression whenever it is needed. Will discuss with @acquamarin too and see if I should open an issue on this.

src/include/main/client_context.h Outdated Show resolved Hide resolved
src/main/client_context.cpp Show resolved Hide resolved
Copy link

codecov bot commented Jan 2, 2025

Codecov Report

Attention: Patch coverage is 87.50000% with 52 lines in your changes missing coverage. Please review.

Project coverage is 86.39%. Comparing base (672ae95) to head (c4f579d).
Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
src/storage/wal_replayer.cpp 13.79% 25 Missing ⚠️
src/main/client_context.cpp 93.84% 8 Missing ⚠️
src/main/storage_driver.cpp 0.00% 6 Missing ⚠️
src/processor/map/map_recursive_extend.cpp 0.00% 3 Missing ⚠️
src/binder/bind/bind_file_scan.cpp 0.00% 2 Missing ⚠️
src/function/table/call/show_tables.cpp 33.33% 2 Missing ⚠️
src/binder/bind/bind_graph_pattern.cpp 80.00% 1 Missing ⚠️
src/function/table/call/show_functions.cpp 0.00% 1 Missing ⚠️
src/function/vector_cast_functions.cpp 50.00% 1 Missing ⚠️
src/parser/visitor/standalone_call_rewriter.cpp 88.88% 1 Missing ⚠️
... and 2 more
Additional details and impacted files
@@           Coverage Diff           @@
##           master    #4674   +/-   ##
=======================================
  Coverage   86.39%   86.39%           
=======================================
  Files        1373     1373           
  Lines       58263    58293   +30     
  Branches     7216     7210    -6     
=======================================
+ Hits        50338    50364   +26     
- Misses       7761     7765    +4     
  Partials      164      164           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@ray6080 ray6080 force-pushed the remove-context-from-parser branch from a4d4a02 to 455ed00 Compare January 2, 2025 14:07
Copy link

github-actions bot commented Jan 2, 2025

Benchmark Result

Master commit hash: 672ae9558307787342f3377f032aeec045fd5322
Branch commit hash: bbd998a0babcb322a9a9449f222685bf66f4cb35

Query Group Query Name Mean Time - Commit (ms) Mean Time - Master (ms) Diff
aggregation q24 646.36 661.69 -15.33 (-2.32%)
aggregation q28 11627.23 11829.38 -202.15 (-1.71%)
filter q14 126.22 129.38 -3.16 (-2.44%)
filter q15 134.05 132.20 1.85 (1.40%)
filter q16 307.07 308.03 -0.96 (-0.31%)
filter q17 447.45 459.68 -12.24 (-2.66%)
filter q18 1919.05 1939.56 -20.51 (-1.06%)
filter zonemap-node 89.15 90.84 -1.69 (-1.86%)
filter zonemap-node-lhs-cast 88.82 90.94 -2.12 (-2.33%)
filter zonemap-node-null 84.85 87.30 -2.45 (-2.81%)
filter zonemap-rel 5686.67 5886.44 -199.77 (-3.39%)
fixed_size_expr_evaluator q07 576.87 582.73 -5.86 (-1.00%)
fixed_size_expr_evaluator q08 799.62 800.59 -0.97 (-0.12%)
fixed_size_expr_evaluator q09 805.68 818.39 -12.72 (-1.55%)
fixed_size_expr_evaluator q10 236.80 243.17 -6.37 (-2.62%)
fixed_size_expr_evaluator q11 231.02 233.99 -2.97 (-1.27%)
fixed_size_expr_evaluator q12 226.46 233.46 -7.00 (-3.00%)
fixed_size_expr_evaluator q13 1457.62 1462.12 -4.50 (-0.31%)
fixed_size_seq_scan q23 112.39 114.18 -1.80 (-1.57%)
join q29 642.28 610.39 31.90 (5.23%)
join q30 10567.08 10746.14 -179.07 (-1.67%)
join q31 4.24 5.47 -1.23 (-22.51%)
join SelectiveTwoHopJoin 53.96 51.87 2.09 (4.03%)
ldbc_snb_ic q35 2517.26 2668.56 -151.30 (-5.67%)
ldbc_snb_ic q36 492.03 456.87 35.16 (7.70%)
ldbc_snb_is q32 1.44 6.59 -5.15 (-78.22%)
ldbc_snb_is q33 12.80 12.23 0.57 (4.65%)
ldbc_snb_is q34 0.86 0.88 -0.02 (-1.97%)
multi-rel multi-rel-large-scan 1634.26 1796.25 -161.99 (-9.02%)
multi-rel multi-rel-lookup 8.24 18.26 -10.02 (-54.87%)
multi-rel multi-rel-small-scan 75.24 98.74 -23.50 (-23.80%)
order_by q25 132.01 130.79 1.22 (0.94%)
order_by q26 456.11 462.09 -5.98 (-1.29%)
order_by q27 1448.34 1478.58 -30.24 (-2.05%)
recursive_join recursive-join-bidirection 294.73 282.40 12.34 (4.37%)
recursive_join recursive-join-dense 7386.12 7425.63 -39.50 (-0.53%)
recursive_join recursive-join-path 23951.15 23907.01 44.14 (0.18%)
recursive_join recursive-join-sparse 1051.65 1086.82 -35.17 (-3.24%)
recursive_join recursive-join-trail 7332.17 7341.05 -8.88 (-0.12%)
scan_after_filter q01 172.08 176.69 -4.61 (-2.61%)
scan_after_filter q02 156.96 164.30 -7.34 (-4.47%)
shortest_path_ldbc100 q37 86.50 90.99 -4.49 (-4.93%)
shortest_path_ldbc100 q38 363.99 367.16 -3.17 (-0.86%)
shortest_path_ldbc100 q39 65.62 57.33 8.29 (14.46%)
shortest_path_ldbc100 q40 446.52 378.75 67.77 (17.89%)
var_size_expr_evaluator q03 2064.19 2097.33 -33.13 (-1.58%)
var_size_expr_evaluator q04 2210.71 2255.33 -44.62 (-1.98%)
var_size_expr_evaluator q05 2708.02 2572.64 135.38 (5.26%)
var_size_expr_evaluator q06 1335.25 1354.31 -19.06 (-1.41%)
var_size_seq_scan q19 1442.41 1458.33 -15.92 (-1.09%)
var_size_seq_scan q20 2646.81 2702.44 -55.64 (-2.06%)
var_size_seq_scan q21 2276.43 2309.07 -32.64 (-1.41%)
var_size_seq_scan q22 125.13 126.72 -1.59 (-1.25%)

Copy link

github-actions bot commented Jan 2, 2025

Benchmark Result

Master commit hash: 672ae9558307787342f3377f032aeec045fd5322
Branch commit hash: 572258a3ce3ed883194ddc7dcfd31e9deb4662b0

Query Group Query Name Mean Time - Commit (ms) Mean Time - Master (ms) Diff
aggregation q24 644.62 661.69 -17.07 (-2.58%)
aggregation q28 11317.21 11829.38 -512.16 (-4.33%)
filter q14 127.94 129.38 -1.43 (-1.11%)
filter q15 128.47 132.20 -3.73 (-2.82%)
filter q16 305.80 308.03 -2.23 (-0.72%)
filter q17 443.65 459.68 -16.03 (-3.49%)
filter q18 1916.84 1939.56 -22.72 (-1.17%)
filter zonemap-node 89.72 90.84 -1.12 (-1.23%)
filter zonemap-node-lhs-cast 88.63 90.94 -2.32 (-2.55%)
filter zonemap-node-null 85.36 87.30 -1.95 (-2.23%)
filter zonemap-rel 5699.31 5886.44 -187.13 (-3.18%)
fixed_size_expr_evaluator q07 574.23 582.73 -8.50 (-1.46%)
fixed_size_expr_evaluator q08 804.75 800.59 4.16 (0.52%)
fixed_size_expr_evaluator q09 815.43 818.39 -2.96 (-0.36%)
fixed_size_expr_evaluator q10 238.69 243.17 -4.47 (-1.84%)
fixed_size_expr_evaluator q11 230.86 233.99 -3.13 (-1.34%)
fixed_size_expr_evaluator q12 225.86 233.46 -7.60 (-3.25%)
fixed_size_expr_evaluator q13 1464.16 1462.12 2.04 (0.14%)
fixed_size_seq_scan q23 117.22 114.18 3.03 (2.66%)
join q29 617.64 610.39 7.26 (1.19%)
join q30 10099.41 10746.14 -646.73 (-6.02%)
join q31 6.95 5.47 1.48 (27.00%)
join SelectiveTwoHopJoin 54.23 51.87 2.37 (4.56%)
ldbc_snb_ic q35 2481.39 2668.56 -187.17 (-7.01%)
ldbc_snb_ic q36 515.77 456.87 58.90 (12.89%)
ldbc_snb_is q32 3.10 6.59 -3.49 (-52.90%)
ldbc_snb_is q33 10.52 12.23 -1.71 (-13.98%)
ldbc_snb_is q34 0.92 0.88 0.04 (4.89%)
multi-rel multi-rel-large-scan 1329.52 1796.25 -466.74 (-25.98%)
multi-rel multi-rel-lookup 6.31 18.26 -11.95 (-65.43%)
multi-rel multi-rel-small-scan 86.30 98.74 -12.44 (-12.60%)
order_by q25 135.27 130.79 4.48 (3.43%)
order_by q26 455.55 462.09 -6.55 (-1.42%)
order_by q27 1454.52 1478.58 -24.05 (-1.63%)
recursive_join recursive-join-bidirection 287.58 282.40 5.19 (1.84%)
recursive_join recursive-join-dense 7430.78 7425.63 5.15 (0.07%)
recursive_join recursive-join-path 23968.84 23907.01 61.83 (0.26%)
recursive_join recursive-join-sparse 1056.64 1086.82 -30.18 (-2.78%)
recursive_join recursive-join-trail 7332.83 7341.05 -8.22 (-0.11%)
scan_after_filter q01 170.73 176.69 -5.96 (-3.38%)
scan_after_filter q02 156.62 164.30 -7.68 (-4.68%)
shortest_path_ldbc100 q37 88.69 90.99 -2.29 (-2.52%)
shortest_path_ldbc100 q38 334.84 367.16 -32.32 (-8.80%)
shortest_path_ldbc100 q39 62.35 57.33 5.03 (8.77%)
shortest_path_ldbc100 q40 447.74 378.75 68.99 (18.21%)
var_size_expr_evaluator q03 2060.99 2097.33 -36.33 (-1.73%)
var_size_expr_evaluator q04 2232.07 2255.33 -23.25 (-1.03%)
var_size_expr_evaluator q05 2688.40 2572.64 115.75 (4.50%)
var_size_expr_evaluator q06 1337.66 1354.31 -16.64 (-1.23%)
var_size_seq_scan q19 1443.12 1458.33 -15.21 (-1.04%)
var_size_seq_scan q20 2633.00 2702.44 -69.44 (-2.57%)
var_size_seq_scan q21 2333.32 2309.07 24.25 (1.05%)
var_size_seq_scan q22 125.76 126.72 -0.96 (-0.76%)

@ray6080 ray6080 merged commit 5ec9fee into master Jan 2, 2025
24 of 25 checks passed
@ray6080 ray6080 deleted the remove-context-from-parser branch January 2, 2025 15:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants