From b7ce5574dead7f7c53fe6362ac1655d8d54fcd0c Mon Sep 17 00:00:00 2001 From: couchot Date: Thu, 25 Jun 2015 22:31:46 +0200 Subject: [PATCH] expe ANN --- chaosANN.tex | 116 +++++++++++++++++++++++++-------------------------- main.pdf | Bin 1975663 -> 1976136 bytes 2 files changed, 57 insertions(+), 59 deletions(-) diff --git a/chaosANN.tex b/chaosANN.tex index cb1ce2a..22d5258 100644 --- a/chaosANN.tex +++ b/chaosANN.tex @@ -198,77 +198,75 @@ tandis que le graphe $\textsc{giu}(g)$ l'est. L'entrée du réseau est une paire de la forme $(x,(S^t)^{t \in \Nats})$ et sa sortie correspondante est de la forme $\left(F_{h_u}(S^0,x), \sigma((S^t)^{t \in - \Nats})\right)$ comme définie à l'équationà l'équation~(\ref{eq:sch:unaire}). - - - -Firstly, let us focus on how to memorize configurations. Two distinct -translations are proposed. In the first case, we take one input in -$\Bool$ per component; in the second case, configurations are -memorized as natural numbers. A coarse attempt to memorize -configuration as natural number could consist in labeling each -configuration with its translation into decimal numeral system. -However, such a representation induces too many changes between a -configuration labeled by a power of two and its direct previous -configuration: for instance, 16~(10000) and 15~(01111) are close in a -decimal ordering, but their Hamming distance is 5. This is why Gray -codes~\cite{Gray47} have been preferred. - -Secondly, let us detail how to deal with strategies. Obviously, it is -not possible to translate in a finite way an infinite strategy, even -if both $(S^t)^{t \in \Nats}$ and $\sigma((S^t)^{t \in \Nats})$ belong -to $\{1,\ldots,n\}^{\Nats}$. Input strategies are then reduced to -have a length of size $l \in \llbracket 2,k\rrbracket$, where $k$ is a -parameter of the evaluation. Notice that $l$ is greater than or equal -to $2$ since we do not want the shift $\sigma$~function to return an -empty strategy. Strategies are memorized as natural numbers expressed -in base $n+1$. At each iteration, either none or one component is -modified (among the $n$ components) leading to a radix with $n+1$ -entries. Finally, we give an other input, namely $m \in \llbracket -1,l-1\rrbracket$, which is the number of successive iterations that -are applied starting from $x$. Outputs are translated with the same -rules. - -To address the complexity issue of the problem, let us compute the -size of the data set an ANN has to deal with. Each input vector of an -input-output pair is composed of a configuration~$x$, an excerpt $S$ -of the strategy to iterate of size $l \in \llbracket 2, k\rrbracket$, -and a number $m \in \llbracket 1, l-1\rrbracket$ of iterations that -are executed. - -Firstly, there are $2^n$ configurations $x$, with $n^l$ strategies of -size $l$ for each of them. Secondly, for a given configuration there -are $\omega = 1 \times n^2 + 2 \times n^3 + \ldots+ (k-1) \times n^k$ -ways of writing the pair $(m,S)$. Furthermore, it is not hard to -establish that + \Nats})\right)$ comme définie à l'équation~(\ref{eq:sch:unaire}). + +On s'intéresse d'abord aux différentes manières de +mémoriser des configurations. On en considère deux principalement. +Dans le premier cas, on considère une entrée booléenne par élément +tandis que dans le second cas, les configurations sont mémorisées comme +des entiers naturels. Dans ce dernier cas, une approche naïve pourrait +consister à attribuer à chaque configuration de $\Bool^n$ +l'entier naturel naturel correspondant. +Cependant, une telle représentation rapproche +arbitrairement des configurations diamétralement +opposées dans le $n$-cube comme une puissance de +deux et la configuration immédiatement précédente: 10000 serait modélisée +par 16 et et 01111 par 15 alros que leur distance de Hamming est 15. +De manière similaire, ce codage éloigne des configurations qui sont +très proches: par exemple 10000 et 00000 ont une distance de Hamming +de 1 et sont respectivement représentées par 16 et 0. +Pour ces raisons, le codage retenu est celui des codes de Gray~\cite{Gray47}. + +Concentrons nous sur la traduction de la stratégie. +Il n'est naturellement pas possible de traduire une stragtégie +infinie quelconque à l'aide d'un nombre fini d'éléments. +On se restreint donc à des stratégies de taille +$l \in \llbracket 2,k\rrbracket$, où $k$ est un parametre défini +initialement. +Chaque stratégie est mémorisée comme un entier naturel exprimé en base +$n+1$: à chaque itération, soit aucun élément n'est modifié, soit un +élément l'est. +Enfin, on donne une dernière entrée: $m \in \llbracket +1,l-1\rrbracket$, qui est le nombre d'itérations successives que l'on applique +en commençant à $x$. +Les sorties (stratégies et configurations) sont mémorisées +selon les mêmes règles. + +Concentrons nous sur la complexité du problèmew. +Chaque entrée, de l'entrée-sortie de l'outil est un triplet +composé d'une configuration $x$, d'un extrait $S$ de la stratégie à +itérer de taille $l \in \llbracket 2, k\rrbracket$ et d'un nombre $m \in \llbracket 1, l-1\rrbracket$ d'itérations à exécuter. +Il y a $2^n$ configurations $x$ et $n^l$ stratégies de +taille $l$. +De plus, pour une configuration donnée, il y a +$\omega = 1 \times n^2 + 2 \times n^3 + \ldots+ (k-1) \times n^k$ +manières d'écrire le couple $(m,S)$. Il n'est pas difficile d'établir que \begin{equation} \displaystyle{(n-1) \times \omega = (k-1)\times n^{k+1} - \sum_{i=2}^k n^i} \nonumber \end{equation} -then +donc \begin{equation} \omega = \dfrac{(k-1)\times n^{k+1}}{n-1} - \dfrac{n^{k+1}-n^2}{(n-1)^2} \enspace . \nonumber \end{equation} -\noindent And then, finally, the number of input-output pairs for our -ANNs is +\noindent +Ainsi le nombre de paire d'entrée-sortie pour les réseaux de neurones considérés +est $$ 2^n \times \left(\dfrac{(k-1)\times n^{k+1}}{n-1} - \dfrac{n^{k+1}-n^2}{(n-1)^2}\right) \enspace . $$ -For instance, for $4$ binary components and a strategy of at most -$3$~terms we obtain 2304~input-output pairs. +Par exemple, pour $4$ éléments binaires et une stratégie d'au plus +$3$~termes on obtient 2304 couples d'entrée-sorties. -\subsection{Experiments} +\subsection{Expérimentations} \label{section:experiments} - -To study if chaotic iterations can be predicted, we choose to train -the multilayer perceptron. As stated before, this kind of network is -in particular well-known for its universal approximation property -\cite{Cybenko89,DBLP:journals/nn/HornikSW89}. Furthermore, MLPs have -been already considered for chaotic time series prediction. For -example, in~\cite{dalkiran10} the authors have shown that a -feedforward MLP with two hidden layers, and trained with Bayesian -Regulation back-propagation, can learn successfully the dynamics of -Chua's circuit. +On se focalise dans cette section sur l'entraînement d'un perceptron +multi-couche pour apprendre des itérations chaotiques. Ce type de réseau +ayant déjà été évalué avec succès dans la prédiction de +séries chaotiques temporelles. En effet, les auteurs de~\cite{dalkiran10} +ont montré qu'un MLP pouvait apprendre la dynamique du circuit de Chua. +Ce réseau avec rétropropagation est composé de deux couches +et entrainé à l'aide d'une propagation arrière Bayesienne. In these experiments we consider MLPs having one hidden layer of sigmoidal neurons and output neurons with a linear activation diff --git a/main.pdf b/main.pdf index 9d028a2d87f0b66989be544694332d9da8a32770..a12c2b1a50c97e4d0a95d5d276ef11f9000bc191 100644 GIT binary patch delta 10781 zcmajBWl$Vk(5B7cZo%E%b#Ms5-C=MI1b1g9Fad%)!QI{6C3w)_?ryR-mC;}7_iUdW5qCiohXi#)01{4#D1;uW}xo`&v zq`EF*vVa$iOcPg|alj8H>Nzoj*W8bCaq`7jW#@$#jXz0Ezpg0U^d=th{>G(`)ZFcC zH0|nGLXgp%{#nnNeJn<%fDKR35xKs}0rPYDO?jf*|E6S>(KM){J`Jzzlj@Yy=7d{3 z!QZKywZi_g<#UUBt^D&l&KB^~jUnSRf6;7N90{1@COP}3)-Pxz55_>El~>eulNo+dA#J;j5n0hAtfiRj0grvOOpHt|qJSae#NArOwUl#|M&< z7G|NH)P+Imd&3C)BF`Jhpu7?-d1G*eyBEy1YA@^6c*nA=?P1KXr!%g+mgE?VW^aZv zVz+2H5U|dN-~9sl_=SJ#MoO-ZSZKRIi8Ht3(UYx&V9T)hEG%9);UOK-9ACu1o6{Dp z)!D^OHMWMA>q;W-QfZ(4n+xF{!&vaNeVb;!CnoQzv(zHr+u)>-Y&6J9L^A_FZx~$d zK;Wswp77j1nNV}Mpf@sqjJB|tt!28t82+L!!t~>;^Lb@J+aUkF*umc;UJ)VFn`=4k z7TlGhOQ)ARV50IBwJ;<&+m;q$4(hO~D`9QQb4=e}%U+T-Onz%t>^7ecsow|{ ze{5$vUYY!yOkcq=<(U?|E%w_Bt{L1q7m3BaFK8E^lLKAGd-4*#ZLf$g;UKKclmDqo z-VthC$(l@HfEn6T;W-^5WjVj9VYza5QmXwmt4~`SfOn?eSIBayee*&jB%qTPlgOJL z1o8~jVLM9sv5e>rYS=nFJ#YRdcxs~|&~cJWQ8>~Di`tNfVE|Pyn&mI1lm%nhXk8Q` z(rczkb_y5K5|=vmoV4I%8`h%Qu0ZZ4qFapFxzvT81CpXx;~upUv(GIvi@s4gRf&JD z*@N?=-%)gm_)gUm)1luVD_Y{ z!u0?K_GxEk<_UiiqjePDrUqw{+1&g zB|I>@<&Lc1EM0Xrs#?$jY8Bc(dW-Hd1j$e(JM$@Rg(zFlhpUTZb-~tJFA)S5u3+db7)4Cq#>Y{s?gZ#$J2WVW}$2q^YN*P7nB7@EV z4-7AB6)}muZ9$d7Oe3slsT$U0_|sA+zzw!VP}vSnYP54BNeOE-E&b4XZWc-D-KC*Q z*OO?z^IxGvk)BEX7I>vVbsu|T|Hk4*G`KGV)4-Zz62?;n9|j!MV!{bmI(CPUW688* z`k4vUB?{mzTO&wcAoe&oGTIQ$^qtug(7t;$(gwE|>L0x3(m?gpyPlx5g1QvTJBA;4 z{YYZOU;h2yweUY#P>ne(Dpz7++4+37I8qPi&9QB3fpZh;8n^{`xkgkjDU-EJ2Iy_L z!}6U?>Ue7gg|`2_7cQC|efW3R-hzcO(MW}N%Ir{tt3>_xXzVQ8}V zvfYG_r1djtl!v*wP<(WUR=56~^y{GyS`;}MS4;HLi|0_#C{H5drR|2!6=t3 z#A%4646irgf=9DOW{KgoP7;F9r=Oeu2_Ig@E`*str*V5VDZ#dB6f~<6GI21Now4xg zzvkFsGVuZ>x;tsEwKmz_Mx-fGLw-t+7pugJRtO}^{Z{imLLkzN$0C-u87Z^UW48-F zwlZ`Kk0%JInoG>mLfb=hMq_siAZtW7|{q+o?Q)kYwwC*_f7As4UHMH0QUtKPSDG^d+>P zurW*8!@*Ts1ZW~|EPa?T?U|TlhyQSnsjO3x2X7A`bfn>JPZ>XNFIt_lWv>uh zgk@~MJ&~l-hv-dxu`oYPkVvi2Szp_g?o$w8LI6jp(En)zA@LV|*SBwXq;G`7XuL!9 zv_Ed(Kda62nW{NFmGL#@rU@1L$_XJmecn)mH>|~|V|`D2hpX@5#)4`ql>$6J z7lRqQy>3MDf`MU^6rn0Vd;4-{;FjGBue>O_cc0eXwyu*D#VWSPR)%+1q{v{K9@eDB#U?8DyKP{T zRncf^u`xmvZQ0jC(dWG9B@`b8S)T0F;lcIyyf7y6smG>FMEC=Prw#n_gcfi$?U0#7 zo36u-zgR?00p*{n&$b9&ge>vpX0p7RE9Ttj3BtP3A2t?sAX(D~FWEDZRL$Qew%(pt7zOq_~?> zq+yYYUR|Q}2bFLxZn>->5GT6Yjf%i1$?53sR1wt#cPfjg9Ze(fgY}xPa~YIe zpDc*xf&AKZsh<4Z;l(!y`h-FtTZ$7B1_z@O2UFj2^ZfRJZ?KJf0kFW;wRR~eZURg> zL_d5UK8eNsB3KLu;38gR176_|B`$YSXv?`|-aq#Ews0t4F7;s!hQ)?)joGY}Vsk%6 zh`F`p62%QMd!v=LU?6=PkoPHC@%{*Y6-?Y8&|@;L>?mY-Y|$nmRPa)#V5A%E?@kSP z_Jd-5t4-!`uP{zG#;(6QG##>u-vOqUS6c+3u?Y`)U=way%hfm4M#3TQ=6+v<@5 z*{bdC?`x59Xov0|#7A1;&fkx@j`*axf7@T_;=TfNDVvvw6I3TN65lQsY%$BHQnUt z@vExThArwGxy!^Hz?9F}yKEebUb6~6D|`H?5CyxeeL4F0oAV0TVIr0zxr7Dx=X7O6 zX^$3xY0%KqI&a>oFoGT}SqX`;BX_TM`Guv8#swZWF%Iddk2P~MiXlF(wgExKmb3`a-(Da1~xG>8lZ)J8LEv8o_@nK=5cr!;!LRd;y zc^Fzy0pFrZ>D0nZo$OvM%14G;^i<&V^c5m$;$i{mx9uvi3B)8(MOdWbp-KE)R|${= zl3t%CR7Y|!V;F6yG^#pfMR7(*q3q7WEre;yIGyM;2TvAS8EQG~hCQ&_SBQcWv6(S`dHcK?&VU1U5hRMO z`wyBk&$2qVT5I&>J$`iJ9~I_%$Cm!zw|3!Vg?}`HT{`|+@+gVJHmF+hYP(|EQnO7o zsC@y!WmERfr8x>c*`4j-055XJISuL&q}(ai)igYzTa-pxE4%p~3`{x9OgSj*@3N^p z?UQ4l16Y46Zkq1Xb7vduBa{SRz|G+aJ#G(prS2j`{+ivHl6uIUVP~qEnf&?dT(`{Y z8(Fyom@jGnr{mSgl5~|u1hHS+#0rfSLBtuEk=kmTQDGXH9EJ3b2TRmNpRU7|kYK05 zMTaLKpb!DRwE#w=^pglooB%SnuN^S#Cs&}4PK~t3h0j|whxG&lgr>icQxG?OD!2DzPD%rT7@I)Pt5sM**r2l z%(PlxVpmV(g&pVC4KhqbLj$?qCU1Mj!CRA~eL3}5#4If%h#qiZh#99?72c*_R62mW zH*9O)!RMayq(M1UcC%y-OtvioHj=Y0A#W;(-}7(71kvtaKBD-Mf{gvv+1;vj|9>ywjo_F%?L3?NtItw|St${c1D5^5tiM$9r4ZSwW+wG9-Nj~L2D_FQ2)hyJV9{41KrpX5KhdFU&G{HDr(RtdrMU!A z(kTKH2`Acq&jxS!mWxv!l{Vb_%c_3O?$uZNc|Wnz*&<6;_Z32Z#YM5DfKLGFDa&Z4 z$i6!lxcVpHb6uuo>M?@0J2nvVVnCscep`3&=GAD{OYBhF{-fh)2I?oW@Kny$6xd(x2ccsfUFsiKFvplGLW@wp!m7gI&?&Ot;&{S1jjF!=F(E)NfJ; z4A~cYih-f$W!Y05?NWcn)Pmkj!wM2Wu{Yxo_t%;mqvj{V6+@%#OR$1Zw;2sE<@6B4 zI;3z5I2kj{_^L0DA0!J=co={8>4IC_xB?~-C}$zMW?bh~G2aEcOG#m37x>AaKCu{Bk>kn)*1YP`)&6vwu{o>DdR(JfAdAS z<^6pkmR^bwWDY_>Oy~9M>`GCjvwt+g%l-Z`nGV30jr zE6B~nB2*y@76c_icx-tR^2Xz^64*<6?m9{cNDT3F_l?Xe+w z8>49Nw31jspS69p0^H=hrZAn2OC8Vfo+R;qNoEb(+#|BZGVPmr#Zrmd5=XUmvU2xu zvo?49-^w`^GL4OpS_;Jx002|XGclQ3kbdAup_7XVp+XZPU&ITIn)!|RhyPKWgy~uJk1D~zf*}daJ^bA8{GKD6O^mDK01uq$w1C-Jp0OQey?_f>0iwr1uUHgGT=LId?mclymB>&r zN$Ztm2%_^B40XmBB+rLJ_*HNkaAWoBPUGuZEhjXV&ezB&g}g3OCF#vZ9+UNx3M}d0 zmJlZ)GT(_cZ*3Fpp^9hTaUQSH>%!ay-CHy2j$I3g0VU;#+QAqZve*M5E3|AFVUqbEqpDxp=&HmS~sk(7G(D+^{ZRV!$aVRHh*uEdx-^Z zm6HRT#@-iqxGZ#G*APb#(qsxPDBnPNY@=dex1k}U-t=F32QSmr^0J!T6X8eDBl+xL zYTnVq(5c^ajd)h)D#L<{4p}v2bADUs_;jr_!W`xi)&b63H~xmM!80cqx6*FA=zX=$Ado%51H10|k@gJQe)jF6(0m zF?d^sql1$8xb&&m!6AQ|am^z&iHXD#BHOG2Dmk2y42uUy(z3u4gu-k=%sQYSn7p6j zsQ@%1xF@3gLe)>EHH1?ut!Ui*YZwo^y3-T#&ouf43`8LPTS)(N*1!cmjM35;azmH!kEA$J$C_<2*{wyvYHyCr z6(O_N;On~8+3_>j;_IDt(y(JOr+J1Q`i)1-9IJFFmXMw7O@hmts?6g`BM{kiV1B%`(P)|Jn^+|UK4Y%H|oidpZYx9>@P7Gdsz z+t3KCL*H-w<|lb%g72y2`LT>1;Z*ipX`X8PU;XnVEFj77VW)=b@Q>7j+3DG9Ojm_p zS9z;Vhn~*$jyh!K7o-tt>cIl8Bt zCDfWAWeE)S_p9}Pd^@dwkVBb`h{t{y!(o*KRXbY=#1cvkMe8Z-(;vm=QT;j@Lt~-*+_LR1SAGRY;7Wqjd+7v9SC z4)W7iv){({?^(Z|wU|Dbz6!2fjlNIKaK^5>JcCgM96*sTJ@_>iL+(HR^?V)D^A=QJ zoQxJ4Rj{PkCw7h2cE>d^*b~uq`)wC}GN)9F5fi6v=MB^>e@vSH_qKYw$(%?#D7dH% zNaB|v8X8)tIt*l<*bZ6E#0uiq z@dr!Z;(m)?{F5h8CBvSKO{uOV2xL`qL~T+cY=IwcKQ^q&RVM9=rAuDD`QlrPdmrVwAM7%4_JRH6P{?fsn{uIMrv3vvz|C%e#Ue27rczxCmy{1{%}(ofYs!G4xdK{6l2eye)YuQ+=vgHQLx7Nhl8KvuOk>#G0_N=_YOq zSK*N9bo(^UimQ%c$}b;OQ3i-~t$pU3kB61s3N=AQxKK7qOmIN;iWUiG-g5=p-cTHn z>dAz1H5mq1cJ3l-kpN}c7jPY-jev;lcrDYQCKBqE{@5k_vcISS@pf|udUG#L$JQDMFW2dJU72Z5Gs%|n^8aQ6pf|_wf`J^o?V8cVHnZ$1z**sXuIp{uq z?J&-`TKcNvsf%PEfK>8g{Zy$nT)Z&uIo$vF^dJ9Q-iqgLN|QbzY?!QeEX4hAhszC>R2L;^>_1bdoF1kb(n5q{jiA&71UZWvG&Tbe~# z)soU#<2}r{3feUc(1>S()}6m(%9@=UKU64=(ffO*Qx#0aUUlN`g}q{~@`Y1+#yW!o zg8wLc;wsG=(#nV!O^ZS+WnqiW&-hg;2PAyAyS=|CB9e?k#^4UwB?QdBR^KQtX&%C_ zuz{Henbm>vKJaApi;_;sS42S77_V$&YEgh{0x4sQvJYyZil*VZ(gna=5GZ{?)%F4v z+hS|8+)F|J^Zj;Dc7z@GK3LpsZjS)@r4PZaWptgE3?^jnBCbQp2git}Md|``+nVc( zQ{tLqzj@rd71%L}T2Kv$Iy`dhdx%^Eadq>t^-8&7`9+mV>m7MT(yFTJ$R!O#DIPhe z)uZnOI3l~Hvp8vnQJzIU~7hL`8`D$qvn_N*ZG&)Dsz&+{|? z5^S5NTe~s;J+4sG0(>4+5xWnr@UKiBG{E;ck~yBAo@>8gAsLLEUpcE zLwibhE|@~K#K(hr2&5NrXWSkoD4mJaiM2zs#g3f{2$a}BJGyo?^jkJOZpyf2-niYs zyEIn2o=_`iF@d=TNVdeXLZ&FZVcUHIu&3*G2e!hkvoYpSHy#pLrq5@yc6TAFWI6C2 z>yM4r*)Ph&CunLOJjR0Ja7VS*2-9DqBQ06Z!7ao40P-2|?)S#62mGC4pYOvnaG0Ro zoB%P)Xz)t*4qAugGbGydHCr_WZ&rRsL1#3Y*un_cGKa`?h=`;6TePVn#_Z#cpjPL< z1w2l$p^o_~k;HEzv0|h_kLX|GpXSmmg!PplB9C>i+3jB&J``CWZ}VAl4Tn7Le@)?S zf}4l+*euyIe3NI>EKJzBjhTu$2Orp9Rw>n)FWZSWf3wPW*7w{7 z;#b~+7H5jB!4JhT&9=3cDWp#Xo9*Wyvy``i%+G;)C*Q@@Jnj0zv~yD%H&|rU0Rg2p z3$BlilNU%QykV{R#Ag6{b(jwJmwoXC@M6EOpO!aFRs!5}<8j9V>OIKmIV)=aKq7ue zN;HyAN|&TPX@&DlcGFAf)o7)a?f(!3v>xvqjw<&qklIa z4|JgloX>feiuj|dWy+(_hfIWExrM6UCo|stqTT80g0Bp*HyXF37wlCDx)DHtt_t>v zaap(ZJC_IBsUz@i5&S}MH=`(pU4P8q@4oW#OEmST9kQ3GMosArtVJPU^M1V4xq_*L z=v-k}0t`GtUUt1dyChK!uC3}x`3#+$OmZx_=EnIZV2?9|HDAgLc0nqg?Vofyw7s?#LN+~ZD2>V<{JXUQr0ovPK zz2>8(`|ro3W9AKe5&PXQ#h)jV&8n^X-9G$T7GA6TS>h(E4KL1PnFi@mI22qI+z8DY ztN?7VMhRO9|5Y>&<4gyxNO9gzRKHz<>*-$R>7Q))i4gXBy2f*GdKy+R0Tk*uUJ_i!3}IK{rZY6lFdDvb=PUsdyYHIx7y`(*u#aHRQ!d}(*2^}U<$&OHE51rb&ATx>x+*!K z?Nfa0QG_LA``_*-h}6GbEy>{t_j!yd2!#tlCfuPE=Mmi77Az4=$QDg2JVyvM7rJaO z6A_$VFR&Grp_kuEsJo{ULMw-wkaxU!$`^V7kquPD(r)megnWX{0$;UJ?D ze1JKbg&vifNI}7rT24WM0iGmJ7vb|>U2{StDg`vzutKqldlZbrJ95HARR8+5<|M+r7q3y+ z=IIYR%Wt+OKLoCtDiee<_G~@*HI=8?#TUzTY83loIbN_t@DT0>_wW^N=r$|E{r zS9*gWf;62t+tJ9*Uu=o2k)pw#(`WKD7NfGbo_lMtscoVALzA!C+aBG1T)< zW7{FflwT@~zW^Egg*0OEV=#~L=3~_*EaGYVslYf&q&SLlpy^)N_!Mqzq1hb-8aYt? zG^me`sJ8#7wT>Ug3)k;^qzSuaCc7Vq@-?ZCr^l2KTgNVjPZs#AX67hBYCoU09-uyAs1v?4y9Lw&yF6KjXjXJ~uZ`)3Emtwvq3! zr7itvf;y3*lY2>8=*s=?X|J&!@d5ymw~d$>#?Z)TKCpzS%%vPOIlhyZJVt7;qE z+6t^(iBt)`$x|_^DulxYn+!IrO8-4Y5gZFC*dhWt4OJ29xDa?4Ucaf~WdpjlGV#GD z?E6z8fa$M8ST#(Gf3!}lOa%9(Sj9i=Sz3`pdy!i+30`ASL)%>035@{-y01&8EdNby zAAD)BD{Ihej3T$WFX|f1MC_PbmTts{gsf8D;mA;Ds2zuWbD8DRl37TPBXRzB{9jM=%C|5Zc- zm#N#Ur>lS4M=||@O!?PvhW`kUUBbKpJLAU^+n^1#0NO1~%?}xf`{!5OX3%w$o(>=s zjoWC+W(X`^Mb?d)HW^B;8}J53*cS$Hptvgd5Shhchd_zwzT93xlO1&Qw$`ukdOfKf zv3D}B$W-zR#Hr%_tF)gDjsIeCz?^-;ghj`-h;zZq4GO3ITqbN$v@6o?MP?R>Z3um& z>co63{{f^K#L2Unc%-TX*_{b`oTE>_S_({d6_D8=9|Ab8za4tA>hypsH2>0w@jQd< zKYC*ATVr7;0HUxy@K?xNIK8wGIVm;4yT&d#tW88)FE!JDS0%Rh?qt+IE}Lilg9O;;{kh5D>|;H^jZb@ZPv$|8lFe zd}Y&iW~Nyu-OmF|eCl)iBTDB+qF}mm_6~)7LEpMoSFTByILJiKQ#Z|aR-;R&n9+0I zELm=ITBATG+qbZQp(3DON^Vlu_og|<%V1yPuSesmu;NF``-R?zY+f_ze@Dx-y}r^p z++PE*`%f)cFOT4L$bN?z*0d_&DlQEQAg;Vw3~`x> zahuU|Ot_aaLAC*U4s*0RuNa|E&0XGp2Us-Y;1@_$Po()4VGj1G4n0S?ySLi}|Oe~(Nka9C r*?;*zX0Nj0pBL!3dy)48OEKCDkSaq@L5ar0!^MY6PcNe;i~7F+0g|X) delta 10315 zcmai%Ra6|nvS@L)Awhz>4eqWXxCD21$OM-lGr-^ufx&{iy9E#K8eD?AyM&i>&wBUw ze(YMcs_Ub>t4n%l!ux#08v=!gB0v$LNKj-b3KSKJ21SQrKrx|MP;4j;6c>uu0623) z;O2()P?9l&Pf3m4wwo^3x}^OhMa8He=!UGGoWGnWjy81idfF~4TuDYXj0;6*eJq0= z%RDdqj6zn34`mfgs5HxM}3Y?9m_KscmUB)SH;0 z)7LBiFuAi;r04fjYk2owz|{Xqc$>N)KGG%Ej0W`v{FB2p;B{{=Ll*|LpDnb30WA}E z7dAGFqoR2-mP4$18Ib<|P;MYKP+-sui8DW$Ki_yc+Z$g2rXSr`SCzWFE*z8IN8kmO zjwu&yL#^D~B<^_NVquXS8zv5&?LODVN4UsJ=5pYvxx?PUggZ7GmH%3}~J*@0F?Kv>ojgZrWMJ53|q* zjke{VQR2|^@zYm&#kHJYns)j!MTkMXM=BadT;yz7{~&sGwiwa7J`TLT)$bRjI`hDH zW>CC{fLg+4sK1+Y4}Z(~xhP5!o{Tgb4&MxhV>0*CEBjmEBnI&EAyvVbFU^0(rZ#14G78?g*}OO(&vY<$?kJ1lfGh3xMN)A4$~-KJl1)4G#?YWlrh=Y0Y9R*3t;OJnEimj}k_aP!0+B z51tln@!jNBH);F`Xhm90tT;CXLR!68M|Wv%4c2E8UKWrTnNtQ7p`mjBu23+@^&5@> z)3~eQKq)&h@w%qff>{)5>jM|VOFFplt^q&Y?LlN{>=w>k-=-C&-uJO~>}M!@(|4=V zTC)e#lFv8{Nz)E-44V3Iv}xSIJ>QYw5cm=77*44Yb-CQ}8Db*yd2}V{!x|MQ4}SC1 zpN`w_e&_<-J=d-Ac_b_t3q_)$*Cj)8)0X4&bhk3IxE`lm^HhGjV1I+oiCuxOQlp9S zR`6~-pnK?g$h==9zp2H9!DC>3m(7mL9M<@l6_I5VMr@6awOF~M77x6H8eO);AG`Ms zO>VOjlJZyscGw zn%srStZS>N5X*FDmu!zN$8}ZUs&}t&ML1o?>bU%h=3JjGS~t;pZd&n!lmu~@cEdij zaPtN)3QE%D@L;A_q9D_Uk7W<1HTLupt+1?_d{t{%@ zBsgaGu%{u9quHY;BZi0Kq1|z2r-OzF)!Xy-t8gfKx>r*hChaaH8L3qFQ4;v5I_<{D49S6(nzZ%2oXlB2cU;|#h9vvKJVK{vX4q&h%+3wYv|5;sdPn$yfRG-= zJfe=bk0Q0Y@|VhyRU|1+bPJ%nO~?s@B#2;nMD~8l^f7|}@e8=M^vL%c@cLHfZH=}K zF!Wq@O{Kk{p)1GPku%^4$BZ=6GO*}k`~`ibVz7y<+3cb2bqZmLSjzA4c07W?ZfAp? zIuy=ZCLI&+wC4{Nxsb=nD0v@{N*?hy)2DuOaOV|^*5P&1+U~CT?CLWq(5J0%HgqO~Y9yP~CXL-EN#8e8S$)$!7xIMVBPkI1AQ+T0ffKbc_ zOf&gmh(<;c+p{*BXz_v+)kWbFk$-J|Cs@iq*3ZuK0!&+jm^^ql>L)TySbb@jXmgX5 zyb>kNAw*C2?Yr%x1o7WE!VDKQo8>; zB5t(wzL3@rJ;9?VR0WCsXgRI5TP|(xnaPs8yf&ZdXZy0cueUFJcG-~UI19_9#9l>;f6nLgjR8IEL~1(53%Io#vT~qu5WNj5BHyCq=ZAXn!61%`Snu zA5S~cX=BiX99OwSWadRZU*Qyby`t7vaH2B{COB+r1BU?MOoYA6_FjD|hoYV{O)2ez z+vrd4bccbf;pyevGic6>+c4f5=3`%5iCoNA zUhrL`{{tcIcLLTAN;Mh8g?<_W1bn2@1?EqGW_wKy<-gOES-eur@4GZtiTk%*Zo6fAVeq!l|(!E5GoCY>=qxk+N zarmArU&Xc(uaKDCzU(b+Cq`tX>I9~CH@sV#SLgdb+R~TtG{NlK5 zLLh^xj$E5$b7O9{0)O)K1lnwB9XyfAJ|x z>UK{@Rf)g!Doe@2+!9;IX0zs5P~9Z8fnsk'%1B=C9-q#2`#zEyOhg-0cEhU@!O z@Ce#)%AbLR?bvv(a;`n^w19hTm%ieX52dSRFyqwMuhv{F30Rj1K5h0aw{t*~9LV@P zKrL*6b{UVPi*Sc6Ny35!8&@@B4Mj=|~ylFF>%7D64UHk?8IiigpMKAfI&%rODS zmR36ZroX1;s{fF=C<)ivwiJFA2ImdxYtx00aD9>@k8M7YQnK_!v9> z^+fLgtu>u5PI~asQ@Q4cijk$3F!Rv(ND~g~p!{n?r9z;Jk$74QiH#gWv=^V*)?ZOO zBa8l*d}pcuKwwgMwezeWv|E9(L)M z!dqXr3{vQ?mzTh+G#R|n)YiNUmVT`EzY{R2+{{zB{NAO6$3M#9N1HJ}j+$3w1z*0P zSaUCyPTI^560ewbPTyxT^yY^%(Rtj-);JC}A8ietu;Jx)i7~T!{Cf z5s+mx6!2M{P&?i%SyLY3G7I`u;fI#1Rs+l)h{~HVz2w2Sna>_8sjQ2fRU+ixMuWH{ zLB~OZBPZ{){B*9KHfJexZ@Ufc$2%o%=_UomJ`_ZziOj7$yZ6sWPm4(7$R0|~izA8i z27xg2nV@-+UsGIPu@Wldkiz*z7(L_n!z%#XG<8P@MwId$i|!M!3c8`HTQhB|u$i@~Z*=f-o=n@tLBvmJ=={vS>FdG;UIZQ6?3(hV zobf>%Ju=fhlvxP1hx;Q>{pA@^lK4WnU&xtJIH}$icW%N6&Dp?a4(FVI7edUrAe|$n zlvscpzm1~86)Uhp&pG=o&_|z0=CC1v>#v@zt0yOOrE28Fh4>s2gNLM-E;>SwXFwcf zm*B&|XnFZ1jrQ1kmT}(oKI`8{n)(4|Q^%IztF(!=?C3|ntr)oTlu`p6W&Qjx z#~DT3L9blh{8K~az_XmK^Mk12m+s{V-ViK_XW}Md|14tBS)JphrLPv<(-5i*ClUs5 zSsH>Ps~#%DL8^+HF*?o{x@Lbaj|Y2T7O(QxJ%@5sOEM*1sl>qz!J~sTR9uy%FWUR= z;a7nSl#}!J{<;-|ftl;Ywlsp9vB7wC`TI@w+s(J%(ZMgJs>J+`(8F z;8|uZwVPxY0qxB$jOCfvH}@;0t7{??zn}Oy^rO0n^OA8j-(rO;K2TO`zxKZ(+&MmQ zz>%zMAazB*=0B^w>f-DTG^)MylBcL^Y{&iWPF`6GbnLwZUFJSORNB5o86Ss%${hM(1e%*UrEmZQL{m=>{DeOvOkI;K0L`~>+{#lzH?%#Z{7cXmHJ>T5 zAC%EEcj8zue#GG@k^KTv)^^U>tschk=FqbL3 z_b{JO06)xy5x~;C6bg{UqVN~O026 zr<6%1`_{DZ{DrzMB}GL3S4lT{!IPk8yl5|nq*kV&NJzWseTMRNCj;fyr~JDlE4r&D zc%H$QCiApF4m(oLyQ!)MTzyi()X#`*=9?6sur0L7;;8T89SnF+;8|uTPQ2Iqo$^iuWL)*q?C=+BI?3Hf{|s5~b>?5? zv6#$f>!;Kz#WEuQXiP`NTp}kzRF-})hi1=Ph2PeL8SCo6Ujr8rJYR-mC6+j{H82H* zQ09nnnTuVf<=bC&>Pl_OEtkQm{;2&MqprXM3M8kQrTt?bD z!T_+1IBv#PyRPf#JmD}JAwKWD6nTfzA zO6#CoChr9uL)!d?r;c#CNrjY1zJ8QiGEdrH%TXzD2A25x$L^^LL5dqJy zDH+Z1`$+deg3%cLwxx%U$RaoS@1YZ`&MC0@rBdk(i+gGs1j}cAP;ZhiB+Li?QQScl zd88Cim;k3(=D>f>&oz@YCOg(Gf1W?m?NsMV$RhvYMp9Wmo~-bi!L?9y_j#3(I3a13 z<+ajDFDZpi(Y6ObmKD7VKawOr=*kf7$)N8pTL`@moXHJ(sd?!hyKsgkI0M0xSa)9v)AsaEq#V7CvAz#x7M*D+jz~9 zAwk6;N(b~hj4YtAAFWNO-ER*fHJD>M?ZO3JYZGQ*bkqt+$ZQfP7I&#fTg)kRP0C}o zW+?bu>^!RVw3Y|B+D`G#=^T~qkQf|o>E%4Ud|vohb>tl4rDe)0BUG7Af!C%hyQZ}U z<`2#@Ji5f^lNao$5buw)RVZ^7L3vg%1Y0lVEX$PCv`)XsQPLsRnHDHrnh-?{qOdBOE9Z1GL)xKq3 zW4^JGylco3QR8L!CBAa>dY+iPE2exC5V)d!0JF&|8dJyp6f4)@rpSGn{k-;+7XCkrN!fVDQsaUia5~L@veB1o;8LMOyl!{o3YeS9R!hf_468* zV!bz1x0QP<;)V3Dcxr(-{Rx7EjCpwbc|i7$%;J%8^OO&P9g#{d^eofw7VNvvwn34Scy_ z_lxmaM6OE|^y?4|T&%HYJA2bzWW=t)f zlfohhRQiI^s&eW=xBIEm70tISGv|Uwr1)rhK*Y;bR#kC4_s-rmU64NUcxj7ydWIPg zxH5~K>G3{QGSYVa&m8eQEo=?7OaA`CPoAKi@7#}~^t&>F9K*R^(br&p4)-axk?4}S zy~x7OSPOfwm{Mg5DKNzW$fm`*wkP&6Qgq8bo3%Oetm~au7RE@H(!G50`Nsh^6CH&z zQT5A-y+{dMuDTMRL`?RK>XC2iCfEAd3l7)JTk}9Xd_XlRF(bm z19q{1(p@=yNh_lD#bmRkL5|lGf)x3N*h?DLe3j8o-hSTG?PzHZ%_fJph#RyzT-uK< zy7ysV9WHxO%{bJ9j&K179@1Jjr@ktCTGh!#c_p2IeQtykotXYj_k@GmxEap$m%7MV z(o~~;wQJ2ESEkL6-@<5>(s58Srnd@qA7!zL5r26Y&IHK-Bn;fsa_W3X93 zHS_+gFT!*s!YwXI(9?<0<)qK0s_A;Uby3wXqS_s1Er;gfxWrP@05k~2Wh;9sY@q$P zgdCU?Ug%`lbTmUSG*d!E5Nju^yW3!hw{})WJ5q*L<{t3&@N4o~)ON3(Xa7xtXFqXx z@m#cO)ar;kWq1cX3r_Q{XP+hJP2y395q_~5{z6-UioMg6FpAF~#NaCAssd$M{8SZ_ zsrtEaW%!hSL9zw0A^w=I_{b}WtaW?m2<#Fgs|c1iALF8+D)cbE>}$GpyV>$ZPL#nF{%@I#-USMuj~st zLD@6BR-axI=k!&yR-s<;DK${gD(jyQoRVk!t-jD;v+z}NS1|LJhzt?f_^XW5xmCu`z8ceg=UNyt zrY&YjDHRwRk0+%rkm>s{`g$3rDMlhKY=IqP@lDG}PE|r&S^jG@>5xZ)%tmS!nLxrZ zwO4HX1jSd{e_>e*t+@@ZK~vQdg#q@O70#`JCM z>B0U|Y-$DGg7FrG!2Tmo5!h~<*{&D6=^n{`=U48jiKjR9$0Zw&(mC8?qd}Bh zkq@H*tz!MUO~E(t_24I&AFRv;34KXriKIMzEp!9oT%D0zMt)0mrYhKp{)gw9e%oI) z+F!)9L%vwPkXhcOQsB+Q%DBEBdH1_!2go3o zxBvEN0JTcAmF8oUoW$?Sb&fWJ>M2rp#bLT7aYnRL`=!)u4-kx-#NUR6NM#UQZ58q` zHN!8s>e)+^1mk9mOu;mZ=B?GY<={@jiP?-c#DnjmQ^hJ9l{z(}SMMYy^#b-RnJIX! zAAHA%Qzy6)qfSg*Y)e^bYsUR?*ZcLCKmC|=h<{pzm)qPa@YNCA225tyWZc| z;KTfYo749ks_Le$F+l)M4E7);Nn~5L`-cc`%S~=#RNa=8K}8TG<7MXy2AaUvAm7cl z-6(Jl217SpDnhd)wsP=?91Qnux0JULkwcZvJDtddYz^e=O>sTc`jqp(A_SsvQD*3@ z!30!MRW9HcP~OJc&cDAFHh44}eGUcTOxf9~w;R}M9n#^(B6&vV_~^(1vyu50}AG?Y_!SS8-X=EJz&VP%?)6<9sqZs@`{si;EA zb_F^s$tDyI;L(Gd(v^eJ-;Jfa8~pYEdMxULFT$|M)f21WFlhQJ22P$i8CDp5d-~pc z4CjN71>3Aa3ZwT=c^gZ5s?iS`PlV^Rd*levL~h{`!kp0mTWy>C%XzqF*VJG24>IIe z;^)z~*$tMk@kFJRxBYCEJ|px!#R0ZAi9wBVu%{Yw-IDfb*EJxdg|3Glon7G!!6{`d zNC={YZqWBQMN=%p5^fLLTG{2n*RSOVAJwy0}Yhrqd>`Yma zPgFtpK8$L=LJK)%?jhkpvo6GB+?Zby>hJzm35f;xh=I((%W(IWPZs@2r9 z{O^1jI#RPFqXCRrj$%O)w{4RyLu7I@i35aForA}BYo+T=s9?di0Xsdlf}`YFd4sr` zNeo)bX@W4gUx#vXaKQ(1gOF(b%@Wj1IL034DNNO1@hRL12s|fRXt%yIPA2NFO$1{2 z{7sq`q=9a0PMj)8`KE9SJSYgA6TPULO&xL83}qF?tJ~Na_ur{GanFKL&EQ#s=xeYj zf(U9bes>Emg5id??Igm`jmf)ZVndyMD=tVds=l1tAY&gkgOfl`Bl6KR`~+`_x=zL_jArP6E$rsZuRAQ)SvomkADVl4wE57} zMR>iqIbkkT6jU$>6$Riw0tw9PWp_5U`o;C<2eTZUxnPtjv~)>AC&Y0xkwtieZq;~1 zA9Xm{|A__3ClCFex1s=74m(usDR|)?u6U9&fq=5>w(fu_Y#d1NCZ`}mY!F3G+J!>S z<5!NUie_B9Dqj>Vy+4@M48mHC#vNt(ab&?i6?82|+wp|>2aE<6*2LKxl<9XOOg z$bly`lNjxrNi>*k7ww666af zm4B>r@+Ps{un5@$xXKAYs(!#3oTVZd>~tF?r10BRct3Z10J0m)*IwyuqlvMwN9-he z)R^@*!fwDa4;;M_0QOR6*^m$jM*b6qnpT1~f5k)KdHI~=kL0FL+8ykUD55^rsCMuo zQaC$O0A7Pj&8{K?-j{GjUok3$4nOarFmp!B9KfC^gpQ9awfzI1~{)PQl9DbAcKrXWntYV$yw!P z{}LB#poflFNlBteLs_pC2L!h#N6@MYl>JxHIj1UC|2-Rw>o!l4e@P7YYidN)>_Xoh ze4($8T(9%3u3KvwpJ;nF^6Y)SZbN3sAP^|p&@dNwd`ersnHgX8S-*Z}LA*SZu=uFb z?|u-Y5v35lP$%m`JN&{tKe=@J7Tw1%5oMcfIWLSo**TSED9~#QD#=_cP>)Iu@<)C_EXLq0JM^EDFyM z`aso=@lX~bNYe+%wHbS$%51m_S0+}4AD3{@p-6s{9>be}GuoP;DA%$1m_z}tB|J=+ zUNC%q>&*Ehj^K@Sfwl?griIAC#lcY^i>=Mg-cgA8k>)Ik`Oh~ae!RS~Qd{wDEB&8k zc+vm*H^NC}-!-&+t-Z2P^$L0amw0@A6rk}y0sKm#!)!cL4u>ozwv-<&2M1;mJ5LhE zwH6)m#wH4_c$uri0LKr`db~dJ-m~qhkJ47kF-bV=7K3vT`mTqz2PZ0S4Bd)SwydUDujVhZ#x(W!X7?#lDkZ@gOoXRThRH@+QnxHo`K-!5&1 zmuv0>P(2##=Zk#SOb3J$b-8uM{iSmC8@Ho`%j%E1(aGlxZ(nTP=gj77#mV^$ZU=ll z=B(xmz1+Vy9%)i8O!Jrj1y#qUS3YULVw((aV>W(I;~XBCr~+`b z%aE8tGkoPh!yJ?%_!WHkXJvR&C3q1}_~0eT?x`70%?kc1DlQzUcMpvo!YE`M2H{ZX zPEqQnQ0hj}>Q4XK?YXBjAUul2K+?Rpv{Ht6^3#bA&U3#>?{LjR!XawDuF?i_VTjg6 q6!IVWPxu_OdK4y~IgGzZ&?>my!YCLiC^5MNcz7`A>1EYEVf;U0IG2e4 -- 2.39.5