Skip to content

Commit

Permalink
Add initial examples based on workshop exercises
Browse files Browse the repository at this point in the history 
See:

https://slides.lobid.org/metafacture-2020
https://github.com/hbz/metafacture-flux-examples
  • Loading branch information
@fsteeg
fsteeg committed May 13, 2022
1 parent 1e71173 commit 25a8c2d
Show file tree
Hide file tree
Showing 11 changed files with 39 additions and 1 deletion.
1 change: 1 addition & 0 deletions resources/examples/experimental/XML_repeated_fields
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
https://metafacture.org/playground/?flux=PG_DATA%0A%7Cdecode-xml%0A%7Chandle-generic-xml%0A%7Cfix%28%22nothing%28%29%22%2CrepeatedFieldsToEntities%3D%22true%22%29%0A%7Cflatten%0A%7Cencode-literals%0A%7Cprint%0A%3B&data=%3C%3Fxml+version%3D%221.0%22%3F%3E%0A%3Crecord%3E%0A++++%3Cfield%3Ea%3C/field%3E%0A++++%3Cfield%3Eb%3C/field%3E%0A%3C/record%3E&active-editor=fix
1 change: 1 addition & 0 deletions resources/examples/experimental/sample7_Backlinks/playground.data
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
001@ 01a5001A 01140:22-05-03001B 09999:12-01-08t16:59:02.000001D 01140:22-05-03001U 0utf8001X 00002@ 0Aa003@ 0967656966004A 085-7503-098-1fPp.006U 003,G03,1594007I So0722800585010@ aporcger011@ a2001017A aruasi019@ aXD-BR021A aA @santa Joana dos matadouroshBertolt Brecht. Trad. e apresentação Roberto Schwarz022A/01 aDie @heilige Johanna der Schlachthöferportug.028A 91185147687Tp1VpipAgnd0118514768E1898B1956aBrechtdBertolt028C 91205728937Tp1VpizAgnd0120572893E1938aSchwarzdRobertoBÜbers.033A pSão PaulonCosac e Naify034D a212 S.034I a22 cm034M aIll.036F x169976509849gAdvzi2281944-7YColeção Prosa do mundopSão PauloJCosac e Naifyl6045E a59m810m820m830m839m840m850m860m870m880m890mB047A SFEaH
29 changes: 29 additions & 0 deletions resources/examples/experimental/sample7_Backlinks/playground.flux
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
FLUX_DIR + "playground.data"
|open-file
|as-lines
|decode-pica
|fix("
paste('{to:118514768}authorOf', '_id') # TODO: 118514768 from 028A.9
retain('{to:118514768}authorOf')
")
|stream-to-triples(redirect="true")
|count-triples(countBy="subject")
|@X;

"https://raw.githubusercontent.com/hbz/metafacture-flux-examples/master/sample6/authority-persons.pica"
|open-http
|as-lines
|decode-pica
|fix("
paste('name', '028A.d', '028A.a')
retain('name')
")
|stream-to-triples
|@X;

@X
|wait-for-inputs("2")
|sort-triples(by="subject")
|collect-triples
|encode-formeta(style="verbose")
|print;
1 change: 1 addition & 0 deletions resources/examples/workshop/sample1_PICA_to_JSON
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
https://metafacture.org/playground/?flux=PG_DATA%0A%7C+as-lines%0A%7C+decode-pica%0A%7C+encode-json%28prettyPrinting%3D%22true%22%29%0A%7C+print%3B&data=001@+%1Fa5%1F01-2%1E001A+%1F01100%3A15-10-94%1E001B+%1F09999%3A12-06-06%1Ft16%3A10%3A17.000%1E001D+%1F09999%3A99-99-99%1E001U+%1F0utf8%1E001X+%1F00%1E002@+%1F0Aag%1E003@+%1F0482147350%1E006U+%1F094%2CP05%1E007E+%1F0U+70.16407%1E007I+%1FSo%1F074057548%1E011@+%1Fa1970%1E017A+%1Farh%1E021A+%1FaDie+@Berufsfreiheit+der+Arbeitnehmer+und+ihre+Ausgestaltung+in+vo%CC%88lkerrechtlichen+Vertra%CC%88gen%1FdEine+Grundrechtsbetrachtg%1E028A+%1F9106884905%1F7Tn3%1FAgnd%1F0106884905%1FaProjahn%1FdHorst+D.%1E033A+%1FpWu%CC%88rzburg%1E034D+%1FaXXXVIII%2C+165+S.%1E034I+%1Fa8%1E037C+%1FaWu%CC%88rzburg%2C+Jur.+F.%2C+Diss.+v.+7.+Aug.+1970%1E%0A&active-editor=fix
1 change: 1 addition & 0 deletions resources/examples/workshop/sample2x_Input_field_names
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
https://metafacture.org/playground/?flux=PG_DATA%0A%7C+as-lines%0A%7C+decode-pica%0A%7C+fix%28%22nothing%28%29%22%2CrepeatedFieldsToEntities%3D%22true%22%29%0A%7C+flatten%0A%7C+encode-literals%0A%7C+print%3B&data=001@+%1F01-2%1Fa5%1E001A+%1F01239%3A18-08-11%1E001B+%1F09999%3A05-09-11%1Ft23%3A31%3A44.000%1E001D+%1F01240%3A30-08-11%1E001U+%1F0utf8%1E001X+%1F00%1E002@+%1F0Af%1E003@+%1F01014417392%1E004A+%1Ffkart.%1E006U+%1F011%2CA37%1E007G+%1FaDNB%1F01014417392%1E007I+%1FSo%1F0752937239%1E010@+%1Fager%1E011@+%1Fa2011%1E017A+%1Fara%1Fasf%1E021A+%1Fxtr%1F91014809657%1F7Tp3%1FVpiz%1FAgnd%1F01034622773%1FE1958%1FaLu%CC%88beck%1FdMonika%1FYPersonalwirtschaft+mit+DATEV%1FhMonika+Lu%CC%88beck+%3B+Helmut+Lu%CC%88beck%1FpBodenheim%1FpWien%1FJHerdt%1FRXA-DE%1FS650%1FgAc%1E021B+%1FlTrainerbd.%1E032@+%1Fg11%1Fa1.+Ausg.%1E034D+%1Fa129+S.%1E034M+%1FaIll.%1E047A+%1FSFE%1Famar%1E047A+%1FSERW%1Fasal%1E047I+%1Fu%24%1Fc04%1FdDNB%1Fe1%1E&active-editor=fix
1 change: 1 addition & 0 deletions resources/examples/workshop/sample3_Transform_fields
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
https://metafacture.org/playground/?flux=PG_DATA%0A%7C+as-lines%0A%7C+decode-pica%0A%7C+fix%0A%7C+encode-json%28prettyPrinting%3D%22true%22%29%0A%7C+print%3B&fix=move_field%28%27021A.a%27%2C+%27Title%27%29%0Amove_field%28%[email protected]%27%2C+Year%29%0Apaste%28%27Publisher%27%2C+%27033A.n%27%2C+%27~%3A+%27%2C+%27033A.p%27%29%0Aretain%28Title%2C+Year%2C+Publisher%29&data=001@+%1F01%1Fa5%1E001A+%1F01140%3A08-12-99%1E001B+%1F09999%3A05-01-08%1Ft22%3A57%3A29.000%1E001D+%1F09999%3A99-99-99%1E001U+%1F0utf8%1E001X+%1F00%1E002@+%1F0Aa%1E003@+%1F0958090564%1E004A+%1Ffkart.+%3A+DM+9.70%2C+EUR+4.94%2C+sfr+8.00%2C+S+68.00%1E006U+%1F000%2CB05%2C0285%1E007I+%1FSo%1F076088278%1E011@+%1Fa1999%1E017A+%1Farb%1Fasi%1E019@+%1FaXA-AT%1E021A+%1FaZukunft+Bildung%1FhPolitische+Akademie.+%5BHrsg.+von+Gu%CC%88nther+R.+Burkert-Dottolo+und+Bernhard+Moser%5D%1E028C+%1F9130681849%1F7Tp1%1FVpiz%1FAgnd%1F0130681849%1FE1952%1FaBurkert%1FdGu%CC%88nther+R.%1FBHrsg.%1E033A+%1FpWien%1FnPolit.+Akad.%1E034D+%1Fa79+S.%1E034I+%1Fa24+cm%1E036F+%1Fx299+12%1F9551720077%1FgAdn%1F7Tb1%1FAgnd%1F01040469-7%1FaPolitische+Akademie%1FgWien%1FYPA-Information%1FhPolitische+Akademie%2C+WB%1FpWien%1FJPolitische+Akad.%2C+WB%1Fl99%2C2%1E036F/01+%1Fx12%1F9025841467%1FgAdvz%1Fi2142105-5%1FYAktuelle+Fragen+der+Politik%1FhPolitische+Akademie%1FpWien%1FJPolitische+Akad.+der+O%CC%88VP%1FlBd.+2%1E045E+%1Fa22%1Fd18%1Fm370%1E047A+%1FSFE%1Fata%1E%0A&active-editor=fix
1 change: 1 addition & 0 deletions resources/examples/workshop/sample3x_Lookup_fields
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
https://metafacture.org/playground/?flux=PG_DATA%0A%7C+as-lines%0A%7C+decode-pica%0A%7C+fix%0A%7C+encode-json%28prettyPrinting%3D%22true%22%29%0A%7C+print%3B&fix=move_field%28%27021A.a%27%2C+%27Title%27%29%0Amove_field%28%[email protected]%27%2C+Year%29%0Apaste%28%27Publisher%27%2C+%27033A.n%27%2C+%27~%3A+%27%2C+%27033A.p%27%29%0Aretain%28Title%2C+Year%2C+Publisher%29&data=001@+%1F01%1Fa5%1E001A+%1F01140%3A08-12-99%1E001B+%1F09999%3A05-01-08%1Ft22%3A57%3A29.000%1E001D+%1F09999%3A99-99-99%1E001U+%1F0utf8%1E001X+%1F00%1E002@+%1F0Aa%1E003@+%1F0958090564%1E004A+%1Ffkart.+%3A+DM+9.70%2C+EUR+4.94%2C+sfr+8.00%2C+S+68.00%1E006U+%1F000%2CB05%2C0285%1E007I+%1FSo%1F076088278%1E011@+%1Fa1999%1E017A+%1Farb%1Fasi%1E019@+%1FaXA-AT%1E021A+%1FaZukunft+Bildung%1FhPolitische+Akademie.+%5BHrsg.+von+Gu%CC%88nther+R.+Burkert-Dottolo+und+Bernhard+Moser%5D%1E028C+%1F9130681849%1F7Tp1%1FVpiz%1FAgnd%1F0130681849%1FE1952%1FaBurkert%1FdGu%CC%88nther+R.%1FBHrsg.%1E033A+%1FpWien%1FnPolit.+Akad.%1E034D+%1Fa79+S.%1E034I+%1Fa24+cm%1E036F+%1Fx299+12%1F9551720077%1FgAdn%1F7Tb1%1FAgnd%1F01040469-7%1FaPolitische+Akademie%1FgWien%1FYPA-Information%1FhPolitische+Akademie%2C+WB%1FpWien%1FJPolitische+Akad.%2C+WB%1Fl99%2C2%1E036F/01+%1Fx12%1F9025841467%1FgAdvz%1Fi2142105-5%1FYAktuelle+Fragen+der+Politik%1FhPolitische+Akademie%1FpWien%1FJPolitische+Akad.+der+O%CC%88VP%1FlBd.+2%1E045E+%1Fa22%1Fd18%1Fm370%1E047A+%1FSFE%1Fata%1E%0A&active-editor=fix
1 change: 1 addition & 0 deletions resources/examples/workshop/sample4_Count_values
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
https://metafacture.org/playground/?flux=PG_DATA%0A%7C+as-lines%0A%7C+decode-pica%0A%7C+fix%0A%7C+stream-to-triples%0A%7C+count-triples%28countBy%3D%22object%22%29%0A%7C+template%28%22%24%7Bo%7D+%7C+%24%7Bs%7D%22%29%0A%7C+print%3B&fix=move_field%28%[email protected]%27%2C+%27Edition%27%29%0Aretain%28%27Edition%27%29&data=001@+%1F01-2%1Fa5%1E001A+%1F01239%3A18-08-11%1E001B+%1F09999%3A05-09-11%1Ft23%3A31%3A44.000%1E001D+%1F01240%3A30-08-11%1E001U+%1F0utf8%1E001X+%1F00%1E002@+%1F0Af%1E003@+%1F01014417392%1E004A+%1Ffkart.%1E006U+%1F011%2CA37%1E007G+%1FaDNB%1F01014417392%1E007I+%1FSo%1F0752937239%1E010@+%1Fager%1E011@+%1Fa2011%1E017A+%1Fara%1Fasf%1E021A+%1Fxtr%1F91014809657%1F7Tp3%1FVpiz%1FAgnd%1F01034622773%1FE1958%1FaLu%CC%88beck%1FdMonika%1FYPersonalwirtschaft+mit+DATEV%1FhMonika+Lu%CC%88beck+%3B+Helmut+Lu%CC%88beck%1FpBodenheim%1FpWien%1FJHerdt%1FRXA-DE%1FS650%1FgAc%1E021B+%1FlTrainerbd.%1E032@+%1Fg11%1Fa1.+Ausg.%1E034D+%1Fa129+S.%1E034M+%1FaIll.%1E047A+%1FSFE%1Famar%1E047A+%1FSERW%1Fasal%1E047I+%1Fu%24%1Fc04%1FdDNB%1Fe1%1E&active-editor=fix
1 change: 1 addition & 0 deletions resources/examples/workshop/sample5_Count_patterns
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
https://metafacture.org/playground/?flux=PG_DATA%0A%7C+as-lines%0A%7C+decode-pica%0A%7C+fix%0A%7C+stream-to-triples%0A%7C+count-triples%28countBy%3D%22predicate%22%29%0A%7C+template%28%22%24%7Bs%7D%3A\t+%24%7Bo%7D%22%29%0A%7C+print%3B&fix=if+any_match%28%27060R.a%27%2C+%27%5E\\d%7B2%7D\\.\\d%7B2%7D.\\d%7B4%7D%27%29%0A++move_field%28%27060R.a%27%2C+%27birth+%28full%29%27%29%0Aelsif+any_match%28%27060R.a%27%2C+%27%5E\\d%7B2%2C4%7D%27%29%0A++move_field%28%27060R.a%27%2C+%27birth+%28year%29%27%29%0Aelse%0A++add_field%28%27invalid+birth%27%2C+%27other%27%29%0Aend%0Aretain%28%27birth+%28full%29%27%2C+%27birth+%28year%29%27%2C+%27invalid+birth%27%29%0A&data=001A+%1F01240%3A06-06-08%1E001B+%1F09999%3A12-07-08%1Ft01%3A36%3A32.000%1E001D+%1F01240%3A06-06-08%1E001U+%1F0utf8%1E001X+%1F00%1E002@+%1F0Tp3%1E003@+%1F0135646189%1E003U+%1Fahttp%3A//d-nb.info/gnd/135646189%1E004B+%1Fapiz%1E007K+%1Fagnd%1F0135646189%1E007N+%1Fapnd%1F0135646189%1Fvzg%1E008A+%1Faf%1E008B+%1Fav%1E028A+%1FdMichael%1FaHack%1E041R+%1F9040031578%1F7Ts1%1FVsaz%1FAgnd%1F04003157-3%1FaArzt%1F4berc%1E047A/03+%1FeDE-101%1E047A/03+%1FrDE-101%1E047C+%1FSpnd%1Fia%1FaHack%2C+Michael%1F0135646189%1E050G+%1FbDiss.+Medizinische+Fakulta%CC%88t+an+der+Universita%CC%88t+Regensburg%1E060R+%1Fa1976%1F4datl%1E065R+%1FaDingolfing%1F4ortg%1E&active-editor=fix
1 change: 1 addition & 0 deletions resources/examples/workshop/sample6_Enrich
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
https://metafacture.org/playground/?flux=%22https%3A//raw.githubusercontent.com/hbz/metafacture-flux-examples/master/sample6/authority-persons.pica%22%0A%7Copen-http%0A%7Cas-lines%0A%7Cdecode-pica%0A%7Cstream-to-triples%0A%7C@X%3B%0A%0A%22https%3A//raw.githubusercontent.com/hbz/metafacture-flux-examples/master/sample6/wiki-persons.foma%22%0A%7Copen-http%0A%7Cas-lines%0A%7Cdecode-formeta%0A%7Cfix%28%22copy_field%28%27gnd%27%2C+%27_id%27%29%22%29%0A%7Cstream-to-triples%28redirect%3D%22true%22%29%0A%7C@X%3B%0A%0A@X%0A%7Cwait-for-inputs%28%222%22%29%0A%7Csort-triples%28by%3D%22subject%22%29%0A%7Ccollect-triples%0A%7Cencode-json%28prettyPrinting%3D%22true%22%29%0A%7Cprint%3B&active-editor=fix
2 changes: 1 addition & 1 deletion src/clj/metafacture_playground/process.clj
View file
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@

(defn- process-flux [file-path out-path]
(Flux/main (into-array [file-path]))
(println "Processed flux file.")
(println (str "Processed flux file: " file-path))
(slurp out-path))

(defn process [data flux fix morph]
Expand Down

0 comments on commit 25a8c2d

Please sign in to comment.