X-Git-Url: https://bilbo.iut-bm.univ-fcomte.fr/and/gitweb/prng_gpu.git/blobdiff_plain/45ee2a595e584941a3ec05a013d6682d88c42732..50092af11c4cbab1c3048d46875f66476118537c:/prng_gpu.tex?ds=sidebyside diff --git a/prng_gpu.tex b/prng_gpu.tex index 3e1aa83..dc91965 100644 --- a/prng_gpu.tex +++ b/prng_gpu.tex @@ -1,9 +1,44 @@ \documentclass{article} \usepackage[utf8]{inputenc} \usepackage[T1]{fontenc} +\usepackage{fullpage} +\usepackage{fancybox} +\usepackage{amsmath} +\usepackage{amscd} +\usepackage{moreverb} +\usepackage{commath} +\usepackage{algorithm2e} +\usepackage{listings} +\usepackage[standard]{ntheorem} + +% Pour mathds : les ensembles IR, IN, etc. +\usepackage{dsfont} + +% Pour avoir des intervalles d'entiers +\usepackage{stmaryrd} + +\usepackage{graphicx} +% Pour faire des sous-figures dans les figures +\usepackage{subfigure} + +\usepackage{color} + +\newtheorem{notation}{Notation} + +\newcommand{\X}{\mathcal{X}} +\newcommand{\Go}{G_{f_0}} +\newcommand{\B}{\mathds{B}} +\newcommand{\N}{\mathds{N}} +\newcommand{\BN}{\mathds{B}^\mathsf{N}} +\let\sur=\overline + +\newcommand{\alert}[1]{\begin{color}{blue}\textit{#1}\end{color}} \title{Efficient generation of pseudo random numbers based on chaotic iterations on GPU} \begin{document} + +\author{Jacques M. Bahi, Rapha\"{e}l Couturier, and Christophe Guyeux\thanks{Authors in alphabetic order}} + \maketitle \begin{abstract} @@ -14,28 +49,818 @@ This is the abstract Interet des itérations chaotiques pour générer des nombre alea\\ Interet de générer des nombres alea sur GPU +\alert{RC, un petit state-of-the-art sur les PRNGs sur GPU ?} ... -\section{Chaotic iterations} -Présentation des itérations chaotiques +\section{Basic Recalls} +\label{section:BASIC RECALLS} +This section is devoted to basic definitions and terminologies in the fields of topological chaos and chaotic iterations. +\subsection{Devaney's chaotic dynamical systems} + +In the sequel $S^{n}$ denotes the $n^{th}$ term of a sequence $S$ and $V_{i}$ denotes the $i^{th}$ component of a vector $V$. $f^{k}=f\circ ...\circ f$ denotes the $k^{th}$ composition of a function $f$. Finally, the following notation is used: $\llbracket1;N\rrbracket=\{1,2,\hdots,N\}$. + + +Consider a topological space $(\mathcal{X},\tau)$ and a continuous function $f : \mathcal{X} \rightarrow \mathcal{X}$. + +\begin{definition} +$f$ is said to be \emph{topologically transitive} if, for any pair of open sets $U,V \subset \mathcal{X}$, there exists $k>0$ such that $f^k(U) \cap V \neq \varnothing$. +\end{definition} + +\begin{definition} +An element $x$ is a \emph{periodic point} for $f$ of period $n\in \mathds{N}^*$ if $f^{n}(x)=x$.% The set of periodic points of $f$ is denoted $Per(f).$ +\end{definition} + +\begin{definition} +$f$ is said to be \emph{regular} on $(\mathcal{X}, \tau)$ if the set of periodic points for $f$ is dense in $\mathcal{X}$: for any point $x$ in $\mathcal{X}$, any neighborhood of $x$ contains at least one periodic point (without necessarily the same period). +\end{definition} + + +\begin{definition} +$f$ is said to be \emph{chaotic} on $(\mathcal{X},\tau)$ if $f$ is regular and topologically transitive. +\end{definition} + +The chaos property is strongly linked to the notion of ``sensitivity'', defined on a metric space $(\mathcal{X},d)$ by: + +\begin{definition} +\label{sensitivity} $f$ has \emph{sensitive dependence on initial conditions} +if there exists $\delta >0$ such that, for any $x\in \mathcal{X}$ and any neighborhood $V$ of $x$, there exist $y\in V$ and $n > 0$ such that $d\left(f^{n}(x), f^{n}(y)\right) >\delta $. + +$\delta$ is called the \emph{constant of sensitivity} of $f$. +\end{definition} + +Indeed, Banks \emph{et al.} have proven in~\cite{Banks92} that when $f$ is chaotic and $(\mathcal{X}, d)$ is a metric space, then $f$ has the property of sensitive dependence on initial conditions (this property was formerly an element of the definition of chaos). To sum up, quoting Devaney in~\cite{Devaney}, a chaotic dynamical system ``is unpredictable because of the sensitive dependence on initial conditions. It cannot be broken down or simplified into two subsystems which do not interact because of topological transitivity. And in the midst of this random behavior, we nevertheless have an element of regularity''. Fundamentally different behaviors are consequently possible and occur in an unpredictable way. + + + +\subsection{Chaotic iterations} +\label{sec:chaotic iterations} + + +Let us consider a \emph{system} with a finite number $\mathsf{N} \in +\mathds{N}^*$ of elements (or \emph{cells}), so that each cell has a +Boolean \emph{state}. Having $\mathsf{N}$ Boolean values for these + cells leads to the definition of a particular \emph{state of the +system}. A sequence which elements belong to $\llbracket 1;\mathsf{N} +\rrbracket $ is called a \emph{strategy}. The set of all strategies is +denoted by $\mathbb{S}.$ + +\begin{definition} +\label{Def:chaotic iterations} +The set $\mathds{B}$ denoting $\{0,1\}$, let +$f:\mathds{B}^{\mathsf{N}}\longrightarrow \mathds{B}^{\mathsf{N}}$ be +a function and $S\in \mathbb{S}$ be a strategy. The so-called +\emph{chaotic iterations} are defined by $x^0\in +\mathds{B}^{\mathsf{N}}$ and +$$ +\forall n\in \mathds{N}^{\ast }, \forall i\in +\llbracket1;\mathsf{N}\rrbracket ,x_i^n=\left\{ +\begin{array}{ll} + x_i^{n-1} & \text{ if }S^n\neq i \\ + \left(f(x^{n-1})\right)_{S^n} & \text{ if }S^n=i. +\end{array}\right. +$$ +\end{definition} + +In other words, at the $n^{th}$ iteration, only the $S^{n}-$th cell is +\textquotedblleft iterated\textquotedblright . Note that in a more +general formulation, $S^n$ can be a subset of components and +$\left(f(x^{n-1})\right)_{S^{n}}$ can be replaced by +$\left(f(x^{k})\right)_{S^{n}}$, where $kG_{f_0}>> \left(~\mathcal{S}_{10} \times\mathds{B}^{10}, d~\right)\\ + @V{\varphi}VV @VV{\varphi}V\\ +\left( ~\big[ 0, 2^{10} \big[, D~\right) @>>g> \left(~\big[ 0, 2^{10} \big[, D~\right) +\end{CD} +\end{equation*} +\end{theorem} + +\begin{proof} +$\varphi$ has been constructed in order to be continuous and onto. +\end{proof} + +In other words, $\mathcal{X}$ is approximately equal to $\big[ 0, 2^\mathsf{N} \big[$. + + + + + + +\subsection{Study of the chaotic iterations described as a real function} + + +\begin{figure}[t] +\begin{center} + \subfigure[ICs on the interval $(0,9;1)$.]{\includegraphics[scale=.35]{ICs09a1.pdf}}\quad + \subfigure[ICs on the interval $(0,7;1)$.]{\includegraphics[scale=.35]{ICs07a95.pdf}}\\ + \subfigure[ICs on the interval $(0,5;1)$.]{\includegraphics[scale=.35]{ICs05a1.pdf}}\quad + \subfigure[ICs on the interval $(0;1)$]{\includegraphics[scale=.35]{ICs0a1.pdf}} +\end{center} +\caption{Representation of the chaotic iterations.} +\label{fig:ICs} +\end{figure} + + + + +\begin{figure}[t] +\begin{center} + \subfigure[ICs on the interval $(510;514)$.]{\includegraphics[scale=.35]{ICs510a514.pdf}}\quad + \subfigure[ICs on the interval $(1000;1008)$]{\includegraphics[scale=.35]{ICs1000a1008.pdf}} +\end{center} +\caption{ICs on small intervals.} +\label{fig:ICs2} +\end{figure} + +\begin{figure}[t] +\begin{center} + \subfigure[ICs on the interval $(0;16)$.]{\includegraphics[scale=.3]{ICs0a16.pdf}}\quad + \subfigure[ICs on the interval $(40;70)$.]{\includegraphics[scale=.45]{ICs40a70.pdf}}\quad +\end{center} +\caption{General aspect of the chaotic iterations.} +\label{fig:ICs3} +\end{figure} + + +We have written a Python program to represent the chaotic iterations with the vectorial negation on the real line $\mathds{R}$. Various representations of these CIs are given in Figures \ref{fig:ICs}, \ref{fig:ICs2} and \ref{fig:ICs3}. It can be remarked that the function $g$ is a piecewise linear function: it is linear on each interval having the form $\left[ \dfrac{n}{10}, \dfrac{n+1}{10}\right[$, $n \in \llbracket 0;2^{10}\times 10 \rrbracket$ and its slope is equal to 10. Let us justify these claims: + +\begin{proposition} +\label{Prop:derivabilite des ICs} +Chaotic iterations $g$ defined on $\mathds{R}$ have derivatives of all orders on $\big[ 0, 2^{10} \big[$, except on the 10241 points in $I$ defined by $\left\{ \dfrac{n}{10} ~\big/~ n \in \llbracket 0;2^{10}\times 10\rrbracket \right\}$. + +Furthermore, on each interval of the form $\left[ \dfrac{n}{10}, \dfrac{n+1}{10}\right[$, with $n \in \llbracket 0;2^{10}\times 10 \rrbracket$, $g$ is a linear function, having a slope equal to 10: $\forall x \notin I, g'(x)=10$. +\end{proposition} + + +\begin{proof} +Let $I_n = \left[ \dfrac{n}{10}, \dfrac{n+1}{10}\right[$, with $n \in \llbracket 0;2^{10}\times 10 \rrbracket$. All the points of $I_n$ have the same integral prat $e$ and the same decimal part $s^0$: on the set $I_n$, functions $e(x)$ and $x \mapsto s(x)^0$ of Definition \ref{def:e et s} only depend on $n$. So all the images $g(x)$ of these points $x$: +\begin{itemize} +\item Have the same integral part, which is $e$, except probably the bit number $s^0$. In other words, this integer has approximately the same binary decomposition than $e$, the sole exception being the digit $s^0$ (this number is then either $e+2^{10-s^0}$ or $e-2^{10-s^0}$, depending on the parity of $s^0$, \emph{i.e.}, it is equal to $e+(-1)^{s^0}\times 2^{10-s^0}$). +\item A shift to the left has been applied to the decimal part $y$, losing by doing so the common first digit $s^0$. In other words, $y$ has been mapped into $10\times y - s^0$. +\end{itemize} +To sum up, the action of $g$ on the points of $I$ is as follows: first, make a multiplication by 10, and second, add the same constant to each term, which is $\dfrac{1}{10}\left(e+(-1)^{s^0}\times 2^{10-s^0}\right)-s^0$. +\end{proof} + +\begin{remark} +Finally, chaotic iterations are elements of the large family of functions that are both chaotic and piecewise linear (like the tent map). +\end{remark} + + + +\subsection{Comparison of the two metrics on $\big[ 0, 2^\mathsf{N} \big[$} + +The two propositions bellow allow to compare our two distances on $\big[ 0, 2^\mathsf{N} \big[$: + +\begin{proposition} +Id: $\left(~\big[ 0, 2^\mathsf{N} \big[,\Delta~\right) \to \left(~\big[ 0, 2^\mathsf{N} \big[, D~\right)$ is not continuous. +\end{proposition} + +\begin{proof} +The sequence $x^n = 1,999\hdots 999$ constituted by $n$ 9 as decimal part, is such that: +\begin{itemize} +\item $\Delta (x^n,2) \to 0.$ +\item But $D(x^n,2) \geqslant 1$, then $D(x^n,2)$ does not converge to 0. +\end{itemize} + +The sequential characterization of the continuity concludes the demonstration. +\end{proof} + + + +A contrario: + +\begin{proposition} +Id: $\left(~\big[ 0, 2^\mathsf{N} \big[,D~\right) \to \left(~\big[ 0, 2^\mathsf{N} \big[, \Delta ~\right)$ is a continuous fonction. +\end{proposition} + +\begin{proof} +If $D(x^n,x) \to 0$, then $D_e(x^n,x) = 0$ at least for $n$ larger than a given threshold, because $D_e$ only returns integers. So, after this threshold, the integral parts of all the $x^n$ are equal to the integral part of $x$. + +Additionally, $D_s(x^n, x) \to 0$, then $\forall k \in \mathds{N}^*, \exists N_k \in \mathds{N}, n \geqslant N_k \Rightarrow D_s(x^n,x) \leqslant 10^{-k}$. This means that for all $k$, an index $N_k$ can be found such that, $\forall n \geqslant N_k$, all the $x^n$ have the same $k$ firsts digits, which are the digits of $x$. We can deduce the convergence $\Delta(x^n,x) \to 0$, and thus the result. +\end{proof} + +The conclusion of these propositions is that the proposed metric is more precise than the Euclidian distance, that is: + +\begin{corollary} +$D$ is finer than the Euclidian distance $\Delta$. +\end{corollary} + +This corollary can be reformulated as follows: + +\begin{itemize} +\item The topology produced by $\Delta$ is a subset of the topology produced by $D$. +\item $D$ has more open sets than $\Delta$. +\item It is harder to converge for the topology $\tau_D$ inherited by $D$, than to converge with the one inherited by $\Delta$, which is denoted here by $\tau_\Delta$. +\end{itemize} + + +\subsection{Chaos of the chaotic iterations on $\mathds{R}$} +\label{chpt:Chaos des itérations chaotiques sur R} + + + +\subsubsection{Chaos according to Devaney} + +We have recalled previously that the chaotic iterations $\left(\Go, \mathcal{X}_d\right)$ are chaotic according to the formulation of Devaney. We can deduce that they are chaotic on $\mathds{R}$ too, when considering the order topology, because: +\begin{itemize} +\item $\left(\Go, \mathcal{X}_d\right)$ and $\left(g, \big[ 0, 2^{10} \big[_D\right)$ are semiconjugate by $\varphi$, +\item Then $\left(g, \big[ 0, 2^{10} \big[_D\right)$ is a system chaotic according to Devaney, because the semiconjugacy preserve this character. +\item But the topology generated by $D$ is finer than the topology generated by the Euclidian distance $\Delta$ -- which is the order topology. +\item According to Theorem \ref{Th:chaos et finesse}, we can deduce that the chaotic iterations $g$ are indeed chaotic, as defined by Devaney, for the order topology on $\mathds{R}$. +\end{itemize} + +This result can be formulated as follows. + +\begin{theorem} +\label{th:IC et topologie de l'ordre} +The chaotic iterations $g$ on $\mathds{R}$ are chaotic according to the Devaney's formulation, when $\mathds{R}$ has his usual topology, which is the order topology. +\end{theorem} + +Indeed this result is weaker than the theorem establishing the chaos for the finer topology $d$. However the Theorem \ref{th:IC et topologie de l'ordre} still remains important. Indeed, we have studied in our previous works a set different from the usual set of study ($\mathcal{X}$ instead of $\mathds{R}$), in order to be as close as possible from the computer: the properties of disorder proved theoretically will then be preserved when computing. However, we could wonder whether this change does not lead to a disorder of a lower quality. In other words, have we replaced a situation of a good disorder lost when computing, to another situation of a disorder preserved but of bad quality. Theorem \ref{th:IC et topologie de l'ordre} prove exactly the contrary. + + + \section{Efficient prng based on chaotic iterations} -On parle du séquentiel avec des nombres 64 bits +In order to implement efficiently a PRNG based on chaotic iterations it is +possible to improve previous works [ref]. One solution consists in considering +that the strategy used contains all the bits for which the negation is +achieved out. Then in order to apply the negation on these bits we can simply +apply the xor operator between the current number and the strategy. In +order to obtain the strategy we also use a classical PRNG. + +Here is an example with 16-bits numbers showing how the bit operations are +applied. Suppose that $x$ and the strategy $S^i$ are defined in binary mode. +Then the following table shows the result of $x$ xor $S^i$. +$$ +\begin{array}{|cc|cccccccccccccccc|} +\hline +x &=&1&0&1&1&1&0&1&0&1&0&0&1&0&0&1&0\\ +\hline +S^i &=&0&1&1&0&0&1&1&0&1&1&1&0&0&1&1&1\\ +\hline +x \oplus S^i&=&1&1&0&1&1&1&0&0&0&1&1&1&0&1&0&1\\ +\hline + +\hline + \end{array} +$$ + +%% \begin{figure}[htbp] +%% \begin{center} +%% \fbox{ +%% \begin{minipage}{14cm} +%% unsigned int CIprng() \{\\ +%% static unsigned int x = 123123123;\\ +%% unsigned long t1 = xorshift();\\ +%% unsigned long t2 = xor128();\\ +%% unsigned long t3 = xorwow();\\ +%% x = x\textasciicircum (unsigned int)t1;\\ +%% x = x\textasciicircum (unsigned int)(t2$>>$32);\\ +%% x = x\textasciicircum (unsigned int)(t3$>>$32);\\ +%% x = x\textasciicircum (unsigned int)t2;\\ +%% x = x\textasciicircum (unsigned int)(t1$>>$32);\\ +%% x = x\textasciicircum (unsigned int)t3;\\ +%% return x;\\ +%% \} +%% \end{minipage} +%% } +%% \end{center} +%% \caption{sequential Chaotic Iteration PRNG} +%% \label{algo:seqCIprng} +%% \end{figure} + + + +\lstset{language=C,caption={C code of the sequential chaotic iterations based PRNG},label=algo:seqCIprng} +\begin{lstlisting} +unsigned int CIprng() { + static unsigned int x = 123123123; + unsigned long t1 = xorshift(); + unsigned long t2 = xor128(); + unsigned long t3 = xorwow(); + x = x^(unsigned int)t1; + x = x^(unsigned int)(t2>>32); + x = x^(unsigned int)(t3>>32); + x = x^(unsigned int)t2; + x = x^(unsigned int)(t1>>32); + x = x^(unsigned int)t3; + return x; +} +\end{lstlisting} + + + + + +In listing~\ref{algo:seqCIprng} a sequential version of our chaotic iterations +based PRNG is presented. The xor operator is represented by \textasciicircum. This function uses three classical 64-bits PRNG: the +\texttt{xorshift}, the \texttt{xor128} and the \texttt{xorwow}. In the +following, we call them xor-like PRNGSs. These three PRNGs are presented +in~\cite{Marsaglia2003}. As each xor-like PRNG used works with 64-bits and as our PRNG +works with 32-bits, the use of \texttt{(unsigned int)} selects the 32 least +significant bits whereas \texttt{(unsigned int)(t3$>>$32)} selects the 32 most +significants bits of the variable \texttt{t}. So to produce a random number +realizes 6 xor operations with 6 32-bits numbers produced by 3 64-bits PRNG. +This version successes the BigCrush of the TestU01 battery [P. L’ecuyer and + R. Simard. Testu01]. \section{Efficient prng based on chaotic iterations on GPU} -On parle du passage du sequentiel au GPU +In order to benefit from computing power of GPU, a program needs to define +independent blocks of threads which can be computed simultaneously. In general, +the larger the number of threads is, the more local memory is used and the less +branching instructions are used (if, while, ...), the better performance is +obtained on GPU. So with algorithm \ref{algo:seqCIprng} presented in the +previous section, it is possible to build a similar program which computes PRNG +on GPU. + + +\subsection{Naive version for GPU} + +From the CPU version, it is possible to obtain a quite similar version for GPU. +The principe consists in assigning the computation of a PRNG as in sequential to +each thread of the GPU. Of course, it is essential that the three xor-like +PRNGs used for our computation have different parameters. So we chose them +randomly with another PRNG. As the initialisation is performed by the CPU, we +have chosen to use the ISAAC PRNG [ref] to initalize all the parameters for the +GPU version of our PRNG. The implementation of the three xor-like PRNGs is +straightforward as soon as their parameters have been allocated in the GPU +memory. Each xor-like PRNGs used works with an internal number $x$ which keeps +the last generated random numbers. Other internal variables are also used by the +xor-like PRNGs. More precisely, the implementation of the xor128, the xorshift +and the xorwow respectively require 4, 5 and 6 unsigned long as internal +variables. + +\begin{algorithm} + +\KwIn{InternalVarXorLikeArray: array with internal variables of the 3 xor-like PRNGs in global memory\; +NumThreads: Number of threads\;} +\KwOut{NewNb: array containing random numbers in global memory} +\If{threadId is concerned} { + retrieve data from InternalVarXorLikeArray[threadId] in local variables\; + \For{i=1 to n} { + compute a new PRNG as in Listing\ref{algo:seqCIprng}\; + store the new PRNG in NewNb[NumThreads*threadId+i]\; + } + store internal variables in InternalVarXorLikeArray[threadId]\; +} + +\caption{main kernel for the chaotic iterations based PRNG GPU naive version} +\label{algo:gpu_kernel} +\end{algorithm} +According to the available memory in the GPU and the number of threads used +simultenaously, the number of random numbers that a thread can generate inside a +kernel is limited, i.e. the variable \texttt{n} in +algorithm~\ref{algo:gpu_kernel}. For example, if $100,000$ threads are used and +if $n=100$\footnote{in fact, we need to add the initial seed (a 32-bits number)} +then the memory required to store internals variables of xor-like +PRNGs\footnote{we multiply this number by $2$ in order to count 32-bits numbers} +and random number of our PRNG is equals to $100,000\times ((4+5+6)\times +2+(1+100))=1,310,000$ 32-bits numbers, i.e. about $52$Mb. + +All the tests performed to pass the BigCrush of TestU01 succeeded. Different +number of threads have been tested upto $10$ millions. + +\begin{remark} +Algorithm~\ref{algo:gpu_kernel} has the advantage to manipulate independent +PRNGs, so this version is easily usable on a cluster of computer. The only thing +to ensure is to use a single ISAAC PRNG. For this, a simple solution consists in +using a master node for the initialization which computes the initial parameters +for all the differents nodes involves in the computation. +\end{remark} + +\subsection{Improved version for GPU} + +As GPU cards using CUDA have shared memory between threads of the same block, it +is possible to use this feature in order to simplify the previous algorithm, +i.e. using less than 3 xor-like PRNGs. The solution consists in comuting only +one xor-like PRNG by thread, saving in into shared memory and accessing result +of some other threads in the same block of threads. + +\begin{algorithm} + +\KwIn{InternalVarXorLikeArray: array with internal variables of 1 xor-like PRNGs in global memory\; +NumThreads: Number of threads\; +tab1, tab2: Arrays containing permutations\;} + +\KwOut{NewNb: array containing random numbers in global memory} +\If{threadId is concerned} { + retrieve data from InternalVarXorLikeArray[threadId] in local variables\; + offset = threadId\%32; + \For{i=1 to n} { + t=xor-like()\; + shared\_mem[threadId]=(unsigned int)t\; + x = x$\oplus$ (unsigned int) t\; + x = x$\oplus$ (unsigned int) (t>>32)\; + x = x$\oplus$ shared[tab1[offset]]\; + x = x$\oplus$ shared[tab2[offset]]\; + + store the new PRNG in NewNb[NumThreads*threadId+i]\; + } + store internal variables in InternalVarXorLikeArray[threadId]\; +} + +\caption{main kernel for the chaotic iterations based PRNG GPU efficient version} +\label{algo:gpu_kernel2} +\end{algorithm} \section{Experiments} -On passe le BigCrush\\ -On donne des temps de générations sur GPU/CPU\\ -On donne des temps de générations de nombre sur GPU puis on rappatrie sur CPU / CPU ? bof bof, on verra +Differents experiments have been performed in order to measure the generation speed. -\section{Lyapunov} -\section{Conclusion} +First of all we have compared the time to generate X random numbers with both the CPU version and the GPU version. +Faire une courbe du nombre de random en fonction du nombre de threads, éventuellement en fonction du nombres de threads par bloc. + + +\section{Conclusion} +\bibliographystyle{plain} +\bibliography{mabase} \end{document}