Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrading databases #143

Merged
merged 40 commits into from
Jan 5, 2024
Merged
Changes from 1 commit
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
8586c77
Outline of script file for upgrading databases
EdgesFTW Oct 2, 2023
7940529
Merge branch 'illinois-ceesd:main' into upgrading-DBs
EdgesFTW Oct 2, 2023
3c8d436
arg description update
EdgesFTW Oct 2, 2023
5536cce
Untested, but upgrades database.
EdgesFTW Oct 3, 2023
fd49b2f
Removed single quotes
EdgesFTW Oct 3, 2023
5ccf645
Changed default value to show column was not present at time of creation
EdgesFTW Oct 5, 2023
9c2ae81
Added test for upgrading database from V2 to V3
EdgesFTW Nov 9, 2023
e819425
Fixed flake8 and simplified upgrade-db suffix logic
EdgesFTW Nov 9, 2023
6bc2cc0
Merge branch 'main' into upgrading-DBs
EdgesFTW Nov 9, 2023
8d51e7a
Upgrade-db supports ungathered files, and overwrite param added
EdgesFTW Nov 9, 2023
dd88c90
Small comments in upgrade-db
EdgesFTW Nov 9, 2023
840f957
Added upgrade-db test for ungathered files in CI
EdgesFTW Nov 9, 2023
8ca0160
Fixed old print statements in CI
EdgesFTW Nov 9, 2023
a7473df
Renamed stored log version files
EdgesFTW Nov 16, 2023
c159b1f
Isorted upgrade-db
EdgesFTW Nov 16, 2023
194ab42
Autogather can be disabled
EdgesFTW Nov 16, 2023
b7d4557
Upgrade-db now does not add a run-id col to un-gathered files
EdgesFTW Nov 16, 2023
5ca70c5
Refactoring SQL of upgrade-db
EdgesFTW Nov 17, 2023
a4302b1
CI tests upgrades from v2 and v3 in seperate names
EdgesFTW Nov 30, 2023
b98e1a6
Mypy fixed return type
EdgesFTW Nov 30, 2023
c59a35e
Merge branch 'main' into upgrading-DBs
EdgesFTW Nov 30, 2023
6c22d24
Small comment on upgrading V3 DBs
EdgesFTW Nov 30, 2023
68222e0
Primative pytest test for upgrade_db
EdgesFTW Dec 1, 2023
95136fb
Flake8
EdgesFTW Dec 1, 2023
0d5305c
Flake8 isort
EdgesFTW Dec 1, 2023
d13f944
Basic Documentation for
EdgesFTW Dec 5, 2023
0a962b9
fixed typo
EdgesFTW Dec 5, 2023
b42e82f
PR functional changes addressed
EdgesFTW Dec 5, 2023
399cfe2
Added table to upgrade_db autodocs
EdgesFTW Dec 5, 2023
a95e8fb
Isort
EdgesFTW Dec 5, 2023
6769605
Small doc changes from PR
EdgesFTW Dec 15, 2023
0e64ec0
Flake8
EdgesFTW Dec 15, 2023
80d74af
upgrade_db function always uses schema version defined internally
EdgesFTW Dec 17, 2023
83f3814
extend version table
matthiasdiener Jan 5, 2024
32b5c44
change print to logger
matthiasdiener Jan 5, 2024
1f15bd0
clarify schema_version
matthiasdiener Jan 5, 2024
aacadd1
spelling
matthiasdiener Jan 5, 2024
f6b6567
better table
matthiasdiener Jan 5, 2024
8bf7b18
simplify is_gathered
matthiasdiener Jan 5, 2024
1e35b4a
CI updates
matthiasdiener Jan 5, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Fixed flake8 and simplified upgrade-db suffix logic
EdgesFTW committed Nov 9, 2023
commit e819425100b3432b4d7994646992ed63f24ccce3
Binary file added .github/flame1d_lazy_2021.10.21-07.26.58.sqlite
Binary file not shown.
7 changes: 3 additions & 4 deletions bin/upgrade-db
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#!/usr/bin/env python

import sqlite3
import shutil
import sqlite3


def main() -> None:
@@ -17,12 +17,11 @@ def main() -> None:

args = parser.parse_args()

if not args.suffix:
args.suffix = "_upgrade"
suffix = args.suffix or "_upgrade"

for dbfile in args.dbfiles:
# here we do assume we have a gathered dbfile
new_db = upgrade_gathered_db(dbfile, suffix=args.suffix)
new_db = upgrade_gathered_db(dbfile, suffix=suffix)

new_db.commit()

EdgesFTW marked this conversation as resolved.
Show resolved Hide resolved