From 04d2feb9cbeb43885c223511c56373ae45ebdda6 Mon Sep 17 00:00:00 2001 From: Thomas Kipf Date: Fri, 2 Mar 2018 16:28:39 +0100 Subject: [PATCH] Update Readme --- README.md | 55 +++++++++++++++++++++++++++++++++++++++++++++++++----- nri.png | Bin 0 -> 24582 bytes 2 files changed, 50 insertions(+), 5 deletions(-) create mode 100644 nri.png diff --git a/README.md b/README.md index e3402a4..e2d1c4c 100644 --- a/README.md +++ b/README.md @@ -1,12 +1,57 @@ -# NRI -Neural relational inference for interacting systems - pytorch +# Neural relational inference for interacting systems + +This repository contains the official PyTorch implementation of: + +**Neural relational inference for interacting systems.** +Thomas Kipf*, Ethan Fetaya*, Kuan-Chieh Wang, Max Welling, Richard Zemel. +https://arxiv.org/abs/1802.04687 (*: equal contribution)˚ + +![Neural Relational Inference (NRI)](nri.png) + +**Abstract:** Interacting systems are prevalent in nature, from dynamical systems in physics to complex societal dynamics. The interplay of components can give rise to complex behavior, which can often be explained using a simple model of the system's constituent parts. In this work, we introduce the neural relational inference (NRI) model: an unsupervised model that learns to infer interactions while simultaneously learning the dynamics purely from observational data. Our model takes the form of a variational auto-encoder, in which the latent code represents the underlying interaction graph and the reconstruction is based on graph neural networks. In experiments on simulated physical systems, we show that our NRI model can accurately recover ground-truth interactions in an unsupervised manner. We further demonstrate that we can find an interpretable structure and predict complex dynamics in real motion capture and sports tracking data. ### Data generation -cd data +To replicate the experiments on simulated physical data, first generate training, validation and test data by running: +``` +cd data python generate_dataset.py +``` +This generates the springs dataset, use `--simulation charged` for charged particles. + +### Run experiments + +From the project's root folder, simply run +``` +python train.py +``` +to train a Neural Relational Inference (NRI) model on the springs dataset. You can specify a different dataset by modifying the `suffix` argument: `--suffix charged5` will run the model on the charged particle simulation with 5 particles (if it has been generated). + +To train the encoder or decoder separately, run + +``` +python train_enc.py +``` +or + +``` +python train_dec.py +``` +respectively. We provide a number of training options which are documented in the respective training files. -cd .. +Additionally, we provide code for an LSTM baseline (denoted *LSTM (joint)* in the paper), which you can run as follows: +``` +python lstm_baseline.py +``` -That generates the springs dataset, use '--simulation charged' for charged particles. +### Cite +If you make use of this code in your own work, please cite our paper: +``` +@article{kipf2018neural, + title={Neural Relational Inference for Interacting Systems}, + author={Kipf, Thomas and Fetaya, Ethan and Wang, Kuan-Chieh and Welling, Max and Zemel, Richard}, + journal={arXiv preprint arXiv:1802.04687}, + year={2018} +} +``` \ No newline at end of file diff --git a/nri.png b/nri.png new file mode 100644 index 0000000000000000000000000000000000000000..131f160a5f1f76f4b6c202c6165fa8f4ed83724d GIT binary patch literal 24582 zcmbTd1yEg4wjhcV+}+*X-Q6X)UEEzR?(Xgc2?TeC;O_2jArKq_LFcAlchBoz@7L7K zuDZAOIeYEpYi&Kzs>(7*@c8gxU|>jcvXbgxVBkiediN(7&_C<-)qAn5#qS-PyDr;Ri;5rd3EWHG)`cAsk^(jv=o)-Z$wi4w(hTBL&D(Zv6$dQpUii`8y3lap^$o_XH5@5XknlqA(|_ItwYG6 zw!%c^BLnn<*2eLo=|o#$$C#rC`hpe*rTVw})K`;(CrEj|VOqk{HUrh~#6!g{!A$XB zP+)Lipkdr#AYl|>3}Hr7mSWJ8#DBboOcYPu$Du>Zn2eRt^ZxjOotE(x`zI~%gzEe! zEiN7|e$Z0z0ix;s57PRdai%}eO#dL@;WDP<;b!@wn#wKW;9XZ7#qh zk7x`WjMB?yufh1Y8pslG&a%30U|^ zW}t&J2pSAbPz2}MY(@KM$@ z^RYAIGba}jCKUwogAh1axC2Om4)%_2{6Hb{f8gZ@)&IU`At(I@h`XH-x!7NVq&iBf zq!Lc97NlIv+)QR{oUEige9UaztejkIjHK+WY&)XBqLh#bV}KSZzq+Wc?4j&A?d6G&q$e_L4Cm|6b{ z^JZLzu}j)adUTa_5NpsX;`@Yvu1BY z`VUv(2bleh79nysfTx8y`9ELU{6EY8Ur4>IEkG#$(;5E5!v6*4=49#a1#qPb;#j# zkU*kQT43ei;QTcN7&TbjFyUbqxywYf*y5K=c{B+$uIbQZg-h##|sLk zh=LUL^c?3(;P7BRKVRtS?fti>Zqz?;-kgO%ZKN8rKJ`p5V?8Kj(j8@o_H6KTG8=_- z@{tsA@ZuVdCYHR%vZy`Y&{#h{gi}+)AQ}6|KiBn&SplK=ON!M#dZDdf1E_r9W9QOy zB1T%y=Poapes60VC)OUB-+Q@_g=B&_^@T^P5%&psl>`FI<>TBYBn{%k+3coNO3pH99^U= z#tO}I=nl(wEQQ{i$nI9=g}=Y+mixa5Els}}sFhb(8@vluT6PCxhjYH@>CaobLB+>n zmk0<+*fBjl6)tb+{i&8h_3HXYvmqo1?fqLzyToGEvAL@QA+~6HmgE5er)YT`Z#(AM zVW%EW$!?{1`J`TTY+Apq#AZ;sC~kfw#!*V_ad~CJmL6m?FWZvoezfJ;-mtRts`>oX z*iMxn#m_CPYIid)2mV$0Lbf}81U0Rm2QS|(j;&D;#f=pv;XH@hmpr_9I`6O+R;}fF z@2o%RJraqk$iac^FjvG0K_t*xLL-QHVYFkdw&TUOR?^Zzl;83oDnZ2W1Kq3e*Kt}5 zl1|Tlu!!DeI1H53e7q`XemqiXio6(Gn;wQORC{}2>mlo+yqC2#F1*rI(q6OkkZ8h2 zmXsFbH*8iXx4WP$c&(EzQJ|xBT_m^Pv5J#(BstXo#Trd5OP;E>=Z|gQhqp2MsGPtUlDy`BqN4yhcP&o1|~2^abtA6?0|>j<pgJ+j2MNL^QZfGO(tr+ViicNz@LV$G5o`2CYK9`fEwJiO9)Sq8P%18GdnHhO&oSB9MV z<;6(s@Vs0up^ZE3fXziYg(8xNbPwxBH_>q;21?&BQVLf1ov!?SYn%{6u?}a2iU&G{{=Ybb!|`iY)l{!1?8l2*y=CgkVeGqi5EeGO^iywKmw>H;H3 z0h`x;P~WsZp@X6tWMww;sv4=om~?ACXH=om((>#-+!IdUCCK))=U2C?KW81r39?dm z`o0wYyp%H2&s1%eYN?^Ea+2^cixls4IQ^O5PO^zbl1Qv=wFgA+#K2HH+e21fpoPOETaE0wBN3cnSqs&z)HdQM)@_FgDAcXaAz;9s5N9IdRZ$2V?p11Ofb zy^5{&uBfSeOTld)QAAN_Oul{tD-w&k{S^5b2<{1W42U$QHZYBuqpyMeo5IafbUlzN zi38z}S9gOp+IG>@;A%XPP1&J*e3?!J@f0@aD#Z+&V3aNltu>9bxja90_1m9b;Vc#k zKtjwB?IqSGp=xS#_mP$1`X-#1X^UbXxLPcQ4w-vETcps&(ikbJvvc0Sozk($ENjdb zzIGC*J-o)A+SxU=yB+W8v=v)dICdll(X<3uSrkc-ZNE<;lCmJ^3IpnY%Z?=U8Q9-V z2kG0R=EO7N=7hDF|}R&@%Ks2Bgi8dbo?P-e7Vv z>R77@Uz&VZ|J2sf1e+hz=&*SIX@RS1f=D-{q^%M@r_kg+JhFsI7!6qtYppuzin`AI zEB3O2dJ-$s*=^8qCIkm&rMLe2jz(+t#P5v8^dMZZD&t_(-#OTqG%ndp}zJxv${3M@J5+68~^?=^_OnX8> zw6IKNOur{(#b7w1QTwioeY4_!OBch^iAjZHTt;h@E4ENC>He7Hj8{=;z4iW1mrM~1 z7wSVX5M&1De5W#ETZd?WKO-uwTS5s3cQ|;?u=fC-k}ojDEwe7H)q1d|QDe>+3gk*w zJFO`ft!hD%uj|3KXxoZN*ek&{bQOZ%_w=xn7%G!D%6Ts!cE`a2x1cIlnvBR)fNJF% ziI&5jLIF)G3=ZD#fo4z(pQAIeC2hJXjhepnUkcH7o#BJhB4=j;k&KlrEk6bMH_4wi zH4znFtHYg66{B-0Ss_fY5)B&~YCXc)>^<58?Xr(x!nnh>K$-Q7SE5sa<6=ANFF?B; zA6P0JgRm4g7RlW2q1DwV$>*X0#QZ55&TABClMaV2tgHw%IYzqN=+aWV5$DZlNMDf2 zjPwv=@P&JRdQf676?T35p-G=HPPX&@8#&nuG+hy`9@Bpq(l8y*-;vK&>GM+wBVXns zvSy+UtFz)<(U&qY7R8>xKN`qI9}#nr5D3XbDRSJ&*7FeXCC!T~GLJ#OY|IWWi~{~@ zX_azUoT|{|OqJ-)d)9Y-52yWphTnjUmpQag%?3xyA{ynz*|Uij70&JAtU#Y>LC$tl zV6TK^U|`@f!~umR#dmsR9Czl#BKG5Ua=!81QPS}R_fL9!;W#P|L1^=mrd)TK^x!Kr z?fd4g+Wn*3e4|xeb!lq!Z*23SQb4H^=L%u8n5}0xBSth_QEe^q<>h64Q;U=upy)|o ziokho9&4SC&-w!TmmkF*0d9zfJG2-FI!kJh{(7O!p=HC0?%V2z#9;C8sa*>AD>j*5 zQ;c&TH~OZ1NwOX(1tH}~2ZRs6W_0_jO*FzkS|S_h&+3d>*~ z9v10S3w~lb4TqwX?4eji5h5De;5C-%vF2=aHLP(H-?fqqv*+S2Y4S{nC*z{sTMiBm z0xT@3>_gwtPG%M(3JpaqHdnn4_Q2sXhZG=$!%g@PmI(!@sb!jdfyK|$ZF9E%Kv$L) znGz)pndW5(wwJW7f-?WWEjZqLc2&$8|Eo6CwclW>?5vS_ZZnS}C`&lDyQ6fyNZVlU zD9t-cR1;eKOtwP}#W25aXC);KnCA-bRrT~K2W9iNazB6Kq^FlpV*VhJFYi=^T`)B> zi@dN14-bc%44zmil+jaLp!AUIrvFJzS$JP*rv@C!w%%yAaAmHcmHs(uuRW&rlUbV( zcEt$km!GP4KEzs2X}(<3J3r|Z)#5urcAH&c|8rnCs99YZaX%Que zS!=e6_aLQMP%4uvq|jbK)9)0pmA$AhZ)|QJKHVG+2F&zxJ+}{{-C6JLGO^B3ZifV zV?m+xXbr@y+0_Tr??ox(b16?cwN~4 zRp=tBt*#V$n~$$N_Ys=@(jScI9ehOOqBfXoReS~FymsF6&B16H{XC=>?Bxp#_7_M$ zaV+0h@!CmDsb^oNdCO-o8^^vB1h5|{EkpTd#jht+q%_WjmUKtwuh|PS_^PF1yO5sn zKU-HJiQ@Y%Wzybuipu-NNw*Ejo=Cvm0*g8OcY@NzZ$F4gjlTTpOYm*El4~U?DEmC? z27}23_nmd4(NLmZ ze=53eJ&`sfpuUzRKNI&sWE12#^jvw3^adzcrBmp>_n5ru(UxR&rw^ZZnu?S7)w=kt zMO*VE;iXyLVymzWlm;Jl_|)AP z(xsKjwlgg6&;KMTabaR(k{cpFpFt~!kp6src`2C=g0|v=3mFo~iBveAdSgGT?&$FD z#3^-&ZVTuaKl*sprhS^eQhT|fk#0m!tBpH+iAnK8b5_^8D8CKK&CIH6!6X= zRU-_o$$+AAO<(%bqahQZA0%Hjt$`n%9%CAypPT=)1pL-QuX{C($**a6IVOS|2{c7M zusi1sqr$X?MSB4aq^%1n>OgOw5;G#hL zBG$4r9RJ{b@6SUZd^JUek*39kK<=LJr2=@l#}VTwPey7a2nh;~yw(aA4k#-raZU6v zrT{yN!8L@q{n})!N$1luxO<4W(rw@}=OHwkCudgCFLSCu`nLTr%B^~^@hvN48I^k5 z>H*{4siWO{OiSTWrmjDkz;5Qjt8@L#@S=B>9bqvunxv=sjmkhNbD>eW*Jg;)h{VYl z=q%cC#F#gg;9OlR|6;Oe7Yn)II`#Gugm<-Njw`^o=m}o)47=6l@lw z%9!tGUbL^rPV>A4P`ou4u_d1)il?JyBfCwRy* z_VJTNjpxyAFu))a?nkf%_8OG z(GnnHZjEhb%Cif@_>n2zy$W_-w>q*ta*@duSgkDXK)@~72&+)L8{~oVOT9H;$9dRR zXV$aex~I$c<$8J_s=Jp1oue({5IBFIFIj}{R2Z1F)-C9dxxm+w+ON$p%lerf&aGIt zKPDzdQc}Xwvh2P3t22BF0q*gfkdSs+Htr2*%ckh8YAEsi{sGrdeE979j+dIy91-)gRJz!BWU-_rWW20ar#-l{3d{Pu^&#POqO;y9 zrtLJ+-TCrTOs@}4YEP59h4;4%^K~lr#yJS}Z;myI3L;*ENTR=tE{34sLQx&^qmFr; z2F`q254);XO!t@Vk}-0en%}-*E|*mIIOt{W!wnJOGIFDMv3xBn%XfWHG?xc3_|VsH6})948`ngTsT1?r%VF>~m)mStQQw?+nq#Bq(AAF3E0FPoC{9A} z=e+}FB!y96FIbVEot<(%RTA%~kyFT#dG>-8ov7$&wASzw@|sWnppr zfZJi(VOUsLg;@7@E-vzZ@5KnE(*+r#Fb9YggPI3-oPwJC+D#*r=$p`h%MgZDll|UR zd<#`29DRTG>x~iB?oKky6@d3*D`Hh;r3(3uNh!^bi{sGi`lyNwKMR9BM$TGdKs75f zmS!a)g_->&urbP|1zwMM%E*%5`2ca&WdhZyrr6z0MF1&0kdJ`qJ=hGFd!_G;A?Rvm zA%%qko01wakqSJ^_qVB74;8Db%kla}_$PL_p|264g$_p>H1Kw*vX>>#EzgIqQH;ypaUk)rYUn!L z9UWWmg-)V*K8N~U<3i1{(s-gND9Ed-8^j$HniH1F2af?fe(QuvFe^zFMeo6dj?;O_vXEd%9N^x{`ZM?DA7IlZne8cl1 zB9u+5bt0^%Q_a(a$^0X{_Rz@X*=q{f79K#Wkga+&`4p|4niaO8+ce*9O7$$0)1}0) zzjwD`&UH5n*4>k0Z$o(`jXB_0E3ojUB+yGA|oVUEll%HIC#wv4?N9P0iYqg|UfXe?Nc>q>2fF2|K@&v2V9zXxt(gCtlW z@~4xsc=`1?*It(brd~$T8MWxe0Uh}>W!0CnG6yDA-EfLJZnUqAzx_iqm1aJph>6W zkA6D5sow_!Sub{zm(2d4keV!WUY^+8d{pKieb@_Vq5~q$9>|pexFLbU03hJk+FS##!?oFwrsNQSJb+oGef3hbHN&LVzmj-2gLmC+9h}WPWxsVV8+j z0wWRaM3cwurKW+k!$Nf~=@5MGMnn&ea2z4b9OlxU)fKa=T$v&F*Hl?AlS5AbDqUqu za--4*o0b*~jWA#7*gCg7B*xcV?nm*AI&*mn@@*Cb@bkerJu7}_EuHNcCSg41SQcrO ztD`A{#YEtpH1!T9)6Fe_h;j!w7@esf6o@K_b<6cxv4XG9mGq~pdI+E8Nkd$PdVB+? z00CS?x642%ncSKT#CNlZ_IaKYA&c)t>HRR{ALU=J(^iF>kg6QF15&0*xsRPo_+ zK1KM)M$Xt;&nY(c9^_xuJxlh$Ot> z)}5MZZp*hAPb@gNl0x1F#h>r?s6$=n77W&k=Fra07e7CXWi{|&j>a$!9x~A};c0eD zln}r~^SA*MdFc3T28OCeeiI9L`pGZjaZCRydwG7w@Aer2ib*Mj%NO%xBu>3Kf5}@B z-Vzb(9=&2Vgo9;hN@9Psy?#urwzZnWO$(d=7D%(?yP|a{(l-(Z}A71{VzQ7CeL)yB77bBuhmEkZmUHP3Q#+>m zD>;+#j*Ze~E~XTw_<8^Ha~Ww9?idWkD~^n%7BT~}or0b5SYxaqePMOYP-R}-nd^P? z%25H?ZfVu-Gxvk~Ipj-0Z}vr9 z#q*kcMRPh5QEenGO~Rt@Hs6Ylx)Fxy2PiTPaQ2=XxMwfRHiKd`$8)4bMOKrS%ZKtpyQcY|1piIOg?-)&nvvQmps9Si1BGBumNuQ6;23)-iLneA1tFh zILwhB8>*>9G&Nt*SZa|*Qg>g>R%&INy_A%Tas3Z~4@fi|8OwzN`B{9#Q#F(i!JR%( z6!3^jqt=W`vNkT)!fGxlgx<`R9}{$aanq$fR23;-J8rs{xi)}_Koaj>K3?e;@wNPk z9@*|@RK?O2x#wnj#pb1MfEJ`STCLwL-BREK*Yn72YEjeW8lSdG zeUrCf40~aB>F?qRWnn7nbSETolIAN%-q|DYGy-G^JUhHY1e_*(l2X#S_9v@AQCE$0 z^2zC~$JlM(6Yae?-V}N)3LbhVdhO5OZwYM(>R;D!)$ue+o5&8qR&^{UcAfi@kQy55 zf_qq*@G&Bg@kP7+->h4N<8rc{iez>v$NQ^-wiChiFTn(R)28^qSsBzdq?**prVl%m zGrpU&A50UtOkB(Dn@ekUcV<8eSt@{aZPWKB4=Dy0-PAF;sR}LFPmD6pcV6g z36ENPuv{ZTL!BnC3Q!%t7;PhvRj$BK?vr+VdZM`%ocu>mipq!9&@6?RpcL+^_nQ^# zd;0M-yG2LEO&z69rMP1=r))hUky%Gd^Ha`l$urloIg%NTyww{ZkCouC$b803Q(HUt z3vd%e>}l8u>PBnJw>Mx&ft86ewcyGJx84U&_6{|mA?%%2p$1jq)z9nLwz|`fA!kAf z2U`-F-O7oR*cdG+GUJcZ#!>Wsw3jtu-TS^ddBQ4GOeVoMjL~?^MMrbiM~`7MmEzMa z1U2`Wj`Ges;w6W5n80KH1fUOovk%l2mt1l@3|xxPrwqK$B+;qn^6H82mP{37t72qz3)xjfwHk6~kv;_X>Bu0=uWXeixo>Kc;7_^kH)i z67atA?>&g@?IFYz?3RwTYmJvbSE48-j=@euV-zSr%-J3EGuQIoEKP{|HM8JbJY1wI zbRSdbBMqWoS?`-ulN&j}a5yCstbKi3lvo*ixtR@r8vpoHm;4dCOa6tgA^njpQ8>67 zEsbQEpzGn+P7|ZtOkChG6r=|}vygVWiH&v=ryM6DteKhgc}S@8^HA?X2~{v&a=A2= z8IX^bCs|C=2xB($1&_H7sxJ3<;FewB<~QwjXAX}g=#ZUfs8Ix_gcY5a9iu)F=hL&% zx)%MjkXmEq&Mj2=R5iBl^>lD?j@DAE;^mT#|AKe|zB0Fg-Wc#u59w^=Q)FQuX{yrb zg;s6d9_SD;76r554^P8w?Ap_FTcG5;jfi+vIudY{x3Bbs#6)6nBpg!rEaid{%_9Un zoe7HqmSp)!`}rH=%gu8gC4CCd9b9D`^~7 z_t=8(JJNEYKy}F10fkUILgY5AkCVqS zzAN#tu69zrj)0VcoWP))x|2MMQQtu^mK_|opm?@Kt5bQLO1&~Chm#?h2=DGZ`mJWm zn&iub;yv;VJ*IQ|q6|;7sNx>|a6~ zwFf4n2x7=H<^b=ev#`~7<0l75kdWCrQ>qL8{4O8L{a7?EJ~}%eNvPw2Tstp3;QB1P zsXFI&VnSSl>0hF-WPq+Y!FAZ6JT29YU&;mGlo<9$5iOzpR6<503fe1R2r9prXoZw@ zfpHBDDyEgCy+}5LUwCbta;<4)U;O?)0H2a2@!eyDVeCf9+UPcHDPswA{@e7x}W`P-JQ`)S~M$lP6r1UUYbFQq_|9|9}YJ{ zGFTD*W@Rde$x+b1?0Ghx7cb=9-z#$AtCejD(ZO#9d0?#TBQivTZ*U{!1*w6}YhR`c z(NP4>UY1O!ZldmPSwq%2kB{D%rZGq$~C#==}-TUSzRIU*rp&s|?(|DUx#4r?W_i|~|}VT6FR}>SX)*qK!4`@}3ixD3p4T>F{vNKkOW05=Ly4I=$7xb=+n> z89P0RAb7H^Os1J?KdICq4)Jiql{?Jq@ZKmZ>&Z$mMn+xB-s3l{&ea(B4!@zmfe*z4 z%%LYT-azMa__$D(-xcyaD;pRZVQ+Epyk8&&RM9s|AmMLTmd;tQL#ek(b(OxUcL=Dd_v{^p4=n z+Br*UY~0+MO~}zo&Q%h!EbEdP>DRoe7Q16DYzK4lTvXyr!>d`r((bDK>CY$(xxCCu zIhqXSs?trM6*2;39P-KF?C(?cm|E=L%H{C_qDBRDZO9&`O-`=P6JansU$U>~`)5_k zry8ia>$`aYA(W<-Hr;V#mR?`vT&W>#fw<=(%)HP!kk0oWNZIPsV!YDAtTbWk(QVJi zt5ECMZx4ikBb@hlL^ijwMa4x*6?$d4FAepy@YI4%wK;i5H_J*y*7c}gyndBOj4L2i7%#YA zoDC(TsZkqB3mJ6~`?GsAOv#eNq7Y-oEMfi_dNPsb#C2&5?vK$o^cHV(+4ujE^1B)` zBIu^8nI>9hiI=^P)8Sy=gjcSdr>u? z!H4pd!kO(=#E%8h>aIW>^55BYfJ0l1DgzBkba#bdsc+#+ZY}U`sC8u7oxG zP51FdCcjvPGU)g~n|PFB(<8sQT=E%fX0L=PhLvsio38OEWsKxFx4ml_ z+UBXh@+^H>)2#ixLf`P4)_iyU>sw&`b=t&s&Ae~_Y-5^yb+;Ko#%?zr0m*jk3I5`b zX4+Z@3nh3+lQV}%8(ez!#m27PT` zoV=dfk;gWt(5s0O%kgPdz10XS_o%jpno08LRrE#ige+bf)m+}u(@;AKUCHHT>G7pu zSM&8#fhSJPU%IUOGv+mZMO8j>t%TNxCOHilWx4UdZ%kv8-wkQdJ!HPGG0n2(?#)GW=BnnufY zo{ZsCB|RX>T*g(DKI7oi(8P6rBAI$!*)GQ+izZS zZ_32+mwnE;*P-cbWR+V|HBL33w1&`-QanZQfTN^0lID%$vG_{4qXCwqg;Z_H2bga; zZ+Q3#S}^i^m7p1>|QbA+hc7?)KssksP{8?*m~iy+8(n9VN+V2@5Xnzc8( z&(_6eg!S1yG;ED*+Eh$hS3<9&D(dLO63eAfL-lplqB+-OFsOep*z&x#+%og0Vb&OX zw27pd=j+e*VdML3nsc42KbWNLp!>O>(uV9%DGz&Lksuu_wap-9W1D6@i(D7k%uLSy zKDffszJEw@;pV76GpW-3cz6JCQiHjKpMb)It*{`a-DaM-6SlOER|MvubP88N-kr_4vsd8 zp+6wp50Xc!u*&R+IndWDH_EzRNd&RYUK=3rYPPukv=*3vYpqYxhOO(7?L)*NSOV|G zblv}83^b_`7>wvF`e`Sl?UXs3HaX|aZg{{{U)Y?qalC!VRH^JD8bt;m{Dt3X&`}{u zl8%8b`~3Q4F{=VQzFlj5Txg_6*ac;M3{feL!$5*lmk)GBSr0dde$@DN-a?Hx&oX{* zKvdB75p`zWo!ZF#Pu)&$;$%&@geF=Jzrft`YV^vb)^Xq@m63uX!p|S)EM?FRy0Tt_k{l8jXPF`P;To zZOPFouKg@66NcYc`o903zsYev7GwC<%%2~fr2u^xJ`fd z&uoq2ga$%c^)gBVP(aL-)-{%)O!f1QjI-kh$LJbg8?je(6zv1fTq@FRRN_iob+1Np zOF=mhx|p#AUD?>1{5I&P^nhG}nniNE=hbq%`)coX^CCvnWR<8{Lcj|fgMI8?aN z`UAO;vvlk0XytjegtQ@l2F$Nv?_B2L@XeV)#MNw{DkCM8Z>qD$HOI6LbXaxM^4*|u zn~~L5PZo3pYw?9nqjsjEid~g1$8?-z1d2ded2H{4FzRs0_<3L?3vcx9xW_s< z?&}a9TR)5xC=rXNxWU68%1?QA=S3*KT0JVYTc4Lcu|K)}Qrw70Po7j;=JpiN9GQ{E z z6+q#*w?BTR7-I880T27`6)+V!V0o-vvRuFZ#orQU1YdQ8KK}8*EF@}N7^MH z7G>9(zcgZSiId_-mhHwGG5c&P-e|cnpV(lQZ9X@|qX1m=0@Y0pg8k=hoM*uHn?S=w zb||efJGG3MCQoD%`O;;Bl9`a!ZLB`j zV?qoduvPyd((dx{U?ZmlCD{O`&9(en_#>PCtYr{PSP!#9HvVX?NJLC$x5UYP4600i?AKAfqgPE%IF?ytH!&N~F_q}| zcf%%x-N2vXK~O`ltLXRZDo}tNg2;H;Z4|I!;?gKAFTO#2`6li3+$&HGh$vkTd#0Zw2{f!5BR7?t*yEA_|f80&VuI_5l|p3#Y5F} zx#Uz|j_9av#}`oY1Ze5#a9UF#@-St6;o|02>hi%OCT1R6#l0D`KPtzMPY;iilqdhz zW>;Juf>EeEVKP{)huU0@n2!irS`0+3UTW|prvhH?B2`liEN{&=WU%+Tb$_>;Cr}jV z5=9vaE$0LV_hvax9r2@V3P7>QOyj@Mx;Xpz~Ur{(S%% zb+)|;oN8tW4M5r5B>F6ytvljKAGab>;w7}F_EOIg1Kd~U{r%n4MR`VxfG#K0Gx{7& zJMAPbre9o!M#j7}%YND6?~P@gy0|7KX@9vA-GtS-4d}Kf^raShLuV&&lKp9{jqB^G^7(?_cM2J&^q4y7cOY zY_%e*)V#t07gi^~&t7LA4gtv~!uh(gbrpvTxox=hk6+RK9ao!&!yYp8McKexn~41H zS9i5hK$BLEh=TbShn2iv-s}rxJ>*7eBb2*3$uy>w=qyYTOH0i*#xfNreDy=B^*6|j z^Bh)jOf)=-lyCl@(-{RQ%4q(+P9z_Clfd9cXN<-9+4 z$WuJ17C|mKOGaKBGHSJveUOODZX%o;`UqYUL8EdrFg^Zh;3_wrUgxbkd@lMWqn#cddEBqzu-S~*^kkxkFp&`fMP&% z1{&8Z7UHMN(JOg|M+Lq0Q6i2GE}?~Wex`(65+Bya>}6aOCtYrb_!+x;6T7`yQdILE z9bMYFHIVLJr>Hi(s%mIAS?p2}9en0UWMiAHfLNwyb%)*|nWz&-Jptk*Yp&W-{7IxC zoWk8ZX!zK1gC=h5`k*}UUwS7qXnLggE4}{>npN{oEYhPjQs^O8X5)9HPd_ct6Kg>N z%%(5ONEivviuP@Czoi30lFYEFK>&b>DWQs^SJc-2$8S`<@-1 zQCYr0(@;@;aeYXKRG<`Qo)sn@R{R3&{yJ{v${NqMoEl!L77N|s+P_vbGmh2^#q`5>==i2qqari6jfOO0APPusTZ$b6Rhe># zk%}?7>~mdk#fHJ7=n}Do>#5M|+A`~OCL$%BD?>OwaMk+8AS6VcvdC;Ae3}%cm3>3W zjF3{B9f>fmF-vg($&=y+Uf1}7_K$s^vb^=0BLs46#Lj1Y zzMOf5V&TfhIX{@YfBvs3&cdzf_v_;-Ad*w0ndAT^1*BopDJ7#sT0(MkhjfF0fD$95 zTcu+(l0#{foJbEwZ6lw3f8XD8UC;9e+@I@npS#X^pL1RUti@uF5Unyvp$Dz6b7~H{ zy%n~Xt-N_(e>F=AmcsvI)TeN%y<;;AMT$_L9GCM4)d?m9RS;giVSXYYP(ufiN<+Vx z{kqaffT%8IQ^WSfhcRQz;6T%{P`r-=7aFoI?BGh{RnL5bm)LM=kul6Ll51bigKqBI zENhA@CdLe%3jLT*xBf9;^ngS#GRXzO5;;d&^3t0T{_^z1k+<3OwGcj-BGL(F3G2`p z)pI=i=;ODHV~}yVA+&2lbdS~w#`lPZOFp731@>oR$BpY!y=-%A$6TEZGf)*&U#kH0 zv?_H*uopHbZx_twd`lTTB`2pg+_0wl;ax%^Q$jx|`6+7Y$Ev+(zo=D!%xwyaTOT$? zQIzvK%2n!cz1mHL=?#mwQUe6hG$8`|)U<}pvFZ;qqrZKEjFDZEr_6&R%Yga)2?Bz# zGVr|4NAW+$yFYEsuv2W5Vox2OBF|l@0)!Pb6Ih>U-)43i_U39>s}>LLoy=BF{y_%G zD`o8eyz^Jmo%9(Zf#O~niD&oZeZVWK147rZ*6u%l^GEMAV7yJEytL-_L!k`-?a(FC zP`~~AzmfcbCwU_CnW_SPp?R=}`bjKb1L-{@z{+pl5{4&)e?Vlef9+0b;C&kt3&^bJ zdzwzC?o|Ef;5A94(nxG`aSC`+k$-iHo1cb>o3A0)K4y z%OJ5JW{Ds{bC2NLyv6XLws}!37nZkU1AP}LyvI3ZLSIh#mSmFd5kLGk@!E-AJ;*Li zAnr$Yg7eEfk>u>1T%SAklR_im{_m3ff$ALra;ddnP8|Zq(!$>l-TU>BEN4ML?RZpd zrwb6Xv9?wc!X95(^Od-14X5_KD(@vGBI02#6g~O=;&Ao-1-j*m!*5}OsAEltpspPr zznPb>3^DoqMw*O(1E6LHY27 zOQbG`D-NSHTy?pNQe^R@$xijHsMhZgA=)oga(}SFYHXHiPjBGc-4x-GtceOp5m{@(<%gGb*BecpZ)OVoIAB;;BZVt1iTQfqk^^ zauVrs7HjSYxM-Hq%LO{xvWA@gl>gn?&18G(qJO;%QdGBUsw6zR}aO$e2S{P8|LDs3fuoi)?%1 z==3ERZ37;-x#{=t%zBUHW;9A9 zr-wFf0Xq#sz|wFT@E%bURVUkJhwTHZ2LKYWA-h zdTp-t0V}sFx*j6F{#uwJ+nszJ$$K%*xp=HXq_=ZT^Fp$Rj zBaM2HT0jkO`X>?bq}q3&?v(GBaH+g3r{?$d+beeo31b~%-{RA*0S^RSqMO>pyu^Gz z@f8)G_(|f0?)6pavwOd(#n!I)z10R0*X=C!@L=ONbXBf9y4Qj%xg}g|LJI{mathN) zjJk79Q)U@ynFLoxJO;r>vQLRPhd+Qczi^tQ2dVekXf3DqJ3X;(hFfeO`4)6GJla3q zw@w|OFBf!CvU*Ko?Lu|GMTlG!hcsP`-YwQ!qB_`*?S6~c>(PO&#CW4Bj-F)IOF*GX z{pIZ)0Fw0nD(T_y%v-@r4YY_qJYAOpE`;aJiHQZBmTXi0_`OOR&|WiBvmR;_;Z)ih zv`cW&iuz7EA+psxK7x}_q3acFCEw5}J{-QL&%K&>z~?E&#iiI$FGla?MnMNtS2JcT zf5TuncqlCDwTDI9KwmC<#@E)~`THh%%`U;6?#ZP_ooa%;B3HRMJh1??A0Bd5CmNq(zd>}e{ir^)K;Mn zUZA1{Xn$rku`6|yUCNy%jKsD*F&lRk3j14GZWz#|$SqVp<=g$vDyskHc6ba4J1<%}AF&@DY*&1Wp9l$rpo5Y~i7ZP!A)jB>k2}F72=oW z%df#h(}h}-oVd5}c3vrNxeMI!O@JOAyiuU$sW-tH7)IiZ0`A@_XupeR07iI|eYr^ng4O^_ByTdtB-^(8r8W}~h--FM(~Q&BV# zSxkCXSjgH$|Na80M5uT;d+4q#)c<#SbwnrckSQuCw9HvNXgk_QSr|~$KP}&0N{rz@ z8id&|t}go5D#(>H5e+=IZ(q$61X^ycMn;y`SAg@3sOs3Z91m1X9BC>&92-eIxMJIS z10+KCDar&p@B3R?Cp@z=dS=z=L^>Pzxu{;{Z(dgRt9zPt{r+F~`lUkKK3BPBJt4)i z94z$8rU`Za1WT91Hn&z%h!Rnh8;GSHDs_+9e8l4xE@tT{k7e-P8KA28{wHKZ<#*(( zjY4*0_|WEsCtDK%yHA6X4MuvUW@4KwDrq*hXUZoka_*x&)|J0`-;trk^|@)q8cVix z%4<7)knn!+(In~^tV?~M+Q(~teFZ!C6@S|%PG1Xc*vu~2rZi8{xrA(Hto-7%*47O` zk)rRql4cnwt!JpJ8UdJ1{>DsMG&2t(y=g*2gQDQs@1MBGKXAi`%!j_bU!5iwgu9ek z6VgZS*Mq4iZjPge(r*2Ur2Edp&L%&TW02%&zp$1fQVoqOf-)>Xhk>f->u|u{1O6G* z6O9f$mg0Ea5aRZuCDYJe;?v6FeQY&(^yZga5ZjsWJ8W!)5nFqXh0f~VHoM%ouS1V6 zZuS59ZoiSixxCmLAXL-*xRBa~B6>r{q0&Kq3q?PVI1bzO?TT&E&M;6^u%?IFQb(%^ z>xDYIj(e$T2*mlam-D)IyW;@?=XQl9hBvkzEj-Yz2?uPgRAp){i#33|cr+3h^bZ6F zt6i@&|+eyfx1XY2#T<3Mg9{&{L%gMa=b_hrB4G>vGt%lqeN zaGMmfI_92ay6T3D4}JfX{8uk~hW)x@@kg1qVD28n7~WublTj8()luSAZg=cMapQ51 z-uEd>NwUWO-?pvE6a_vs>!-Hy* zuZP7fSNGt!aks;Zl^f==JRipV;4_nN%{TR1AQGI#&3TsJd>PMt%OwsG8V!fTm*F3! zeFbXUQBnyV`MBdvWJ>5IM?gRTsL$kaZRgpmv1}p5g98>x4?=vaWO~4afPjFtzWdD; z1{uxdnr(>Tl$AAu%IoS*{<(i(lOoF%>ob)6sJyb0iWITDYROH2Q~zU?=VK^TvE8UkG>S>QvR$ht*-KDk|{^7iVW} zY~ZU`j5|9!{Z1i&QK;nU>1itrEInNW3&w+;Ml<>Q3Ke6bI|uQl^n9FZhO$TZd5)!V$ZZ5f9TQ_JUqoeaD zk#8O$Jw369Yw7tp=jQ?FfSc>9@82f{as~q7u1ivSn7-a#ZK=A_Qm(zFcBAR2mX?+t zcuY(T@@Q0W=!22@lYnT(B$Zfx(l3Sa?&F_C*G+!x6CLNyII*{)8JB+l9^~Wp!2y_M z9#4E6s;sDxH#E$|@<~fG+1lDh~0=A$m!Tv_XkdCiO13UfMm|Mim^k_ui4Cxqha2&CM|e#xo?+b z#pftmU+;ZI+=auf2S_j10HNKX*T*dKKOy7E$;lBgM82%4F@upsuxJSW>WdRdd}XX7 zdz#^}4GTGcd%1FR#j+fXF)O6SUoCe-5lm<#%MJbV@^XaOt+ocRbggIP@Bkv1i@ymXL1Yp75U|UFlLDx4ByNT)ZX!Eu)euE2&EH0 z#UCQ76K!+@QepONb(A;m0 z$UW&BjUPbmuz!6ZVW71#er{w$BSVs(ZJ8sJhDD`!YDyoNv(cSTez0_Njd?ztXa_P+ zF3b(R#`)pxp>(wxydJ#T$!6?}z21C&(^{@sdB2e6`(`wJ&GWdM06@Nc%nb{oLOj3e z@C(73Qi&M=cWt~7PfW3_Pb*6X9rCfN(@wm7<{^(F% z=bt8D;7b*%`aLJ?FTm3j~=(4;yELo8@~1vxsr~}u#Mi$^?2Ue`xF4g#eVY& zEewuX4qTU9Z*{kTtWdMknRz{!kU4(w+Ygz9-4G0o`{f1?KE&C7JY3k2te-fRG4bK? zEpOt0DY|yhq5|>XEafz(dPoL~lAc(Cce_mkKpMm#KD0?QRPH>z4cQi=~6RolgyQ4QTYQuJ+wR z`=k##IF{2PE%4PAO*7`q{wi9B2+gHmi`;^EA{heur9-~Cn>q> zj^v@FA?)^<63&cA-@to0@Srn74D^Bb`KD-vtiahUwi*BKq>scPA5w=>lk(J+uY_)V zFzUZZ0iUaNg#LN28mYkf{h>W9_-s{FcYAj+)^UmZpnUFXQV~ER)Vo_m^>(Z{G4%4KAgZdp-PLN)%;h)-A{JISY zElj5F;h>aqkk!yBQdfH!!CY;yUW{k8xHy$)q|oe2&N}u$v=Y9xY)M@sNA@ znHLc?Q=nOWzHUt=&v_hozJwJ}ZT}Q)VxQtuE;|(b#dp9yvaSMUFloIe`IC~KG@k`t zw@A!0&*XQB9vX0pCn05QOoUv>*R%&wz8!aePW^^3a0)W6xK9JN*<(2JM`Cclt1K0J z@&_CYKkHTBH3tquLAvY5O4F`XrrCuYTd<&`VOGhb+nZW8UW9JX+Rtz}Tm(%l>4(Y$ zp7#gp4t%Vr^qpi-8Cla2AMm`5@39y)&s$xFO0n>^=RVx!-}PUUv{jj3TGC?)Hr3mhV~GxpVC>bvs9TjfXUKL(q;$<@yFIr#h;-#m7)MP@aWC#zHJnhO$n6?rz#sS@g@`2D-A{ zC6B#7f}yz40uP(M5tWxU`j@f?bCa9VJ_Il(vEJ%q#UXnKqlOSYoZdCH6M50CWMzLowcHOo-=7qqjh(f&#wga$V z_{CTA_TVvR%8#!lzS{L=f-z#Dh9c zBQTNunij?`Aphz@AF5)5a?0u35@`~MJZAbjqZAD;E2VYL+uLF&kix9Nhs1^Y%J?iy zA|``S_@q^jxc>3$Y3BPqZT%M`LR7Q9Ge2K)c$Q1y$jKYCu|X9P8=%g@_^cr|ED(_N zbJaiXfTe;TwM1Sl^`!TST?-)Hz?41I_F|tt+BRuW5uCQ_bLc~ zM4wORX40ZR)+%H@1(&}xkZ-UZrA1rrrZEp|7!i4FB4ik{u$-O zkFLq2HwCux?b?s_o-v@QfVtKit0eC)B`UQ)Xe7L2Gv-%y5icbCGV|mXDT!bQE@{s;>VUB3dOT<~XP8YpMN6Ztbeby40nAxA=z3 zN~*rA3K1)1T=UBba*&`g_zi{4$E1YX$d)Z4LyS{Q=oaLvyt7Zu#n;0WB4l~vX=X*h zQ$e*YI*r8T;L*#xbKov1&uK1Jnu=ql4kc1Rd49y8htXQsPeEYX*7A0`=?qSxtCr$J z!t}$6u58=K43N02@Hk+}uXG!mzTh;?$4&DR_3zm0>W4;ezlf*hu>k`h-IrjPvYB29 zuNS?c*fd6z0bTjIM?;=S!vt}^qHqeRPJZH2J|U6XS4-};s{6Dyc_jVUWVjvXlX$H7 z$Fp~+EjH7S@9i=gpem%Re5f_ z%$^UZsVL-0#XEuakTg^`JrSV&Vn;4ds(fH~c0YIr%&{ZKx(iZ=$;jU>Pqsaj1TvD1 z+2?eM&f0?YW$g2b_w~CHm7;>&(E&@jBp zV13E@?u&ROkDVD=I=gF!rQo|iF6?+wfm;IFQ%lA>0Db=eFIhw*@vb8cf_nefB4e9S z$3r%A#ZpJj^+wCdy9F^$YH5#1Na7lL)K!5~s|Bsh5vDfb4;a6ZaQ-JKe*yS^QTe}M y;NJOvV{|K(R6pLkkh