X-Git-Url: https://bilbo.iut-bm.univ-fcomte.fr/and/gitweb/GMRES2stage.git/blobdiff_plain/448491bcbd3b99bf6aaeed32bae4854925405051..3f095ff7f3fff2553897be9e8dce25d2c3e9298f:/IJHPCN/paper.tex

diff --git a/IJHPCN/paper.tex b/IJHPCN/paper.tex
index 21fa922..999ce37 100644
--- a/IJHPCN/paper.tex
+++ b/IJHPCN/paper.tex
@@ -492,7 +492,7 @@ that the proposed TSIRM converges while the GMRES($m$) does not.
 In this section four kinds of experiments have been performed. First, some experiments on real matrices issued from the sparse matrix florida have been achieved out. Second, some experiments in parallel with some linear problems are reported and analyzed. Third, some experiments in parallÃ¨le with som nonlinear problems are illustrated. Finally some parameters of TSIRM are studied in order to understand their influences.
 
 
-\subsection{Real matrices in sequential}
+\subsection{Real matrices}
 %%ENDNEW
 
 
@@ -608,7 +608,7 @@ However, for parallel applications, all  the preconditioners based on matrix fac
 are  not  available. In  our  experiments, we  have  tested  different kinds  of
 preconditioners, but  as it is  not the subject  of this paper, we  will not
 present results with many preconditioners. In  practice, we have chosen to use a
-multigrid (mg)  and successive  over-relaxation (sor). For  further details  on the
+multigrid (MG)  and successive  over-relaxation (SOR). For  further details  on the
 preconditioners in PETSc, readers are referred to~\cite{petsc-web-page}.
 
 
@@ -621,18 +621,18 @@ preconditioners in PETSc, readers are referred to~\cite{petsc-web-page}.
   nb. cores & precond   & \multicolumn{2}{c|}{FGMRES} & \multicolumn{2}{c|}{TSIRM CGLS} &  \multicolumn{2}{c|}{TSIRM LSQR} & best gain \\ 
 \cline{3-8}
              &                       & Time  & \# Iter.  & Time  & \# Iter. & Time  & \# Iter. & \\\hline \hline
-  2,048      & mg                    & 403.49   & 18,210    & 73.89  & 3,060   & 77.84  & 3,270  & 5.46 \\
-  2,048      & sor                   & 745.37   & 57,060    & 87.31  & 6,150   & 104.21 & 7,230  & 8.53 \\
-  4,096      & mg                    & 562.25   & 25,170    & 97.23  & 3,990   & 89.71  & 3,630  & 6.27 \\
-  4,096      & sor                   & 912.12   & 70,194    & 145.57 & 9,750   & 168.97 & 10,980 & 6.26 \\
-  8,192      & mg                    & 917.02   & 40,290    & 148.81 & 5,730   & 143.03 & 5,280  & 6.41 \\
-  8,192      & sor                   & 1,404.53 & 106,530   & 212.55 & 12,990  & 180.97 & 10,470 & 7.76 \\
-  16,384     & mg                    & 1,430.56 & 63,930    & 237.17 & 8,310   & 244.26 & 7,950  & 6.03 \\
-  16,384     & sor                   & 2,852.14 & 216,240   & 418.46 & 21,690  & 505.26 & 23,970 & 6.82 \\
+  2,048      & MG                    & 403.49   & 18,210    & 73.89  & 3,060   & 77.84  & 3,270  & 5.46 \\
+  2,048      & SOR                   & 745.37   & 57,060    & 87.31  & 6,150   & 104.21 & 7,230  & 8.53 \\
+  4,096      & MG                    & 562.25   & 25,170    & 97.23  & 3,990   & 89.71  & 3,630  & 6.27 \\
+  4,096      & SOR                   & 912.12   & 70,194    & 145.57 & 9,750   & 168.97 & 10,980 & 6.26 \\
+  8,192      & MG                    & 917.02   & 40,290    & 148.81 & 5,730   & 143.03 & 5,280  & 6.41 \\
+  8,192      & SOR                   & 1,404.53 & 106,530   & 212.55 & 12,990  & 180.97 & 10,470 & 7.76 \\
+  16,384     & MG                    & 1,430.56 & 63,930    & 237.17 & 8,310   & 244.26 & 7,950  & 6.03 \\
+  16,384     & SOR                   & 2,852.14 & 216,240   & 418.46 & 21,690  & 505.26 & 23,970 & 6.82 \\
 \hline
 
 \end{tabular}
-\caption{Comparison of FGMRES and TSIRM with FGMRES for example ex15 of PETSc/KSP with two preconditioners (mg and sor) having 25,000 components per core on Juqueen ($\epsilon_{tsirm}=1e-3$, $max\_iter_{kryl}=30$, $s=12$, $max\_iter_{ls}=15$, $\epsilon_{ls}=1e-40$),  time is expressed in seconds.}
+\caption{Comparison of FGMRES and TSIRM with FGMRES for example ex15 of PETSc/KSP with two preconditioners (MG and SOR) having 25,000 components per core on Juqueen ($\epsilon_{tsirm}=1e-3$, $max\_iter_{kryl}=30$, $s=12$, $max\_iter_{ls}=15$, $\epsilon_{ls}=1e-40$),  time is expressed in seconds.}
 \label{tab:03}
 \end{center}
 \end{table*}
@@ -640,7 +640,7 @@ preconditioners in PETSc, readers are referred to~\cite{petsc-web-page}.
 Table~\ref{tab:03} shows  the execution  times and the  number of  iterations of
 example ex15  of PETSc on the  Juqueen architecture. Different  numbers of cores
 are studied  ranging from 2,048 up-to  16,383 with the  two preconditioners {\it
-  mg}  and {\it  sor}.   For those  experiments,  the number  of components  (or
+  MG}  and {\it  SOR}.   For those  experiments,  the number  of components  (or
 unknowns  of  the problems)  per  core  is fixed  at  25,000,  also called  weak
 scaling. This number  can seem relatively small. In  fact, for some applications
 that  need a  lot of  memory, the  number of  components per  processor requires
@@ -776,64 +776,56 @@ taken into account with TSIRM.
 \end{figure}
 
 
-Concerning the  experiments some  other remarks are  interesting.
-\begin{itemize}
-\item We have tested other examples  of PETSc/KSP (ex29, ex45, ex49).  For all these
-  examples,  we have also  obtained similar  gains between  GMRES and  TSIRM but
-  those  examples are  not scalable  with many  cores. In  general, we  had some
-  problems with more than $4,096$ cores.
-\item We have tested many iterative  solvers available in PETSc.  In fact, it is
-  possible to use most of them with TSIRM. From our point of view, the condition
-  to  use  a  solver inside  TSIRM  is  that  the  solver  must have  a  restart
-  feature. More precisely,  the solver must support to  be stopped and restarted
-  without decreasing its convergence. That is  why with GMRES we stop it when it
-  is  naturally restarted (\emph{i.e.}   with $m$  the restart  parameter).  The
-  Conjugate Gradient (CG) and all its variants do not have ``restarted'' version
-  in PETSc,  so they are not efficient.   They will converge with  TSIRM but not
-  quickly because  if we  compare a  normal CG with  a CG  which is  stopped and
-  restarted every  16 iterations (for example),  the normal CG will  be far more
-  efficient.   Some  restarted  CG or  CG  variant  versions  exist and  may  be
-  interesting to study in future works.
-\end{itemize}
-%%%*********************************************************
-%%%*********************************************************
+\begin{figure}[htbp]
+\centering
+  \includegraphics[width=0.5\textwidth]{nb_iter_sec_ex45_curie}
+\caption{Number of iterations per second with ex45 and the same parameters as in Table~\ref{tab:06} (weak scaling)}
+\label{fig:03}
+\end{figure}
 
 
 %%NEW
 
-\subsection{Nonlinear problems in parallel}
+\subsection{Parallel nonlinear problems}
+
+With  PETSc,  linear  solvers  are  used inside  nonlinear  solvers.   The  SNES
+(Scalable Nonlinear  Equations Solvers) module  in PETSc implements easy  to use
+methods,  like  Newton-type, quasi-Newton  or  full  approximation scheme  (FAS)
+multigrid to  solve systems of  nonlinears equations.  As  SNES is based  on the
+Krylov methods of PETSc, it is interesting to investigate if the TSIRM method is
+also efficient  and scalable with non  linear problems. In PETSc,  some examples
+are provided.  An important criteria is the scalability of the initial code with
+classical solvers. Consequently, we have chosen  two of these examples: ex14 and
+ex20.  In ex14, the code solves the  Bratu (SFI - solid fuel ignition) nonlinear
+partial  difference equations  in 3  dimension.  In  ex20, the  code solves  a 3
+dimension radiative transport test problem.  For more details on these examples,
+interested readers are invited to see the code in the PETSc examples.
+
+In Table~\ref{tab:07} we  report the result of our experiments  for the example
+ex14. 
 
 \begin{table*}[htbp]
 \begin{center}
-\begin{tabular}{|r|r|r|r|r|r|r|r|} 
+\begin{tabular}{|r|r|r|r|r|r|} 
 \hline
 
-  nb. cores   & \multicolumn{2}{c|}{FGMRES/ASM} & \multicolumn{2}{c|}{TSIRM CGLS/ASM} & gain& \multicolumn{2}{c|}{FGMRES/HYPRE}   \\ 
-\cline{2-5} \cline{7-8}
-                    & Time  & \# Iter.  & Time  & \# Iter. &        & Time  & \# Iter.   \\\hline \hline
-   512              & 5.54      & 685    & 2.5 &       570 & 2.21   & 128.9 & 9     \\
-   2048             & 14.95     & 1,560  &  4.32 &     746 & 3.48   & 335.7 & 9 \\
-   4096             & 25.13    & 2,369   & 5.61 &   859    & 4.48   & >1000  & -- \\
-   8192             & 44.35   & 3,197   &  7.6  &  1083    &  5.84  & >1000 &  --   \\
+  nb. cores   & \multicolumn{2}{c|}{FGMRES/BJAC} & \multicolumn{2}{c|}{TSIRM CGLS/BJAC} & gain  \\ 
+\cline{2-5}
+                    & Time         & \# Iter.  & Time   & \# Iter. &  \\\hline \hline
+   1024              & 159.52      & 11,584    &  26.34  &     1,563  &  6.06  \\
+   2048             & 226.24       & 16,459    &  37.23 &     2,248   &  6.08\\
+   4096             & 391.21     & 27,794   &  50.93 &   2,911  &  7.69\\
+   8192             & 543.23     & 37,770   &  79.21  &  4,324  & 6.86 \\
 
 \hline
 
 \end{tabular}
-\caption{Comparison of FGMRES  and TSIRM for ex45 of PETSc/KSP with two preconditioner (ASM and HYPRE)  having 25,000 components per core on Curie ($\epsilon_{tsirm}=1e-10$, $max\_iter_{kryl}=30$, $s=12$, $max\_iter_{ls}=15$, $\epsilon_{ls}=1e-40$),  time is expressed in seconds.}
-\label{tab:06}
+\caption{Comparison of FGMRES  and TSIRM for ex14 of PETSc/SNES with a Block Jacobi  preconditioner  having 100,000 components per core on Curie ($\epsilon_{tsirm}=1e-10$, $max\_iter_{kryl}=30$, $s=12$, $max\_iter_{ls}=15$, $\epsilon_{ls}=1e-40$),  time is expressed in seconds.}
+\label{tab:07}
 \end{center}
 \end{table*}
 
 
-\begin{figure}[htbp]
-\centering
-  \includegraphics[width=0.5\textwidth]{nb_iter_sec_ex45_curie}
-\caption{Number of iterations per second with ex45 and the same parameters as in Table~\ref{tab:06} (weak scaling)}
-\label{fig:03}
-\end{figure}
-
-
-
 \begin{table*}[htbp]
 \begin{center}
 \begin{tabular}{|r|r|r|r|r|r|} 
@@ -851,36 +843,46 @@ Concerning the  experiments some  other remarks are  interesting.
 
 \end{tabular}
 \caption{Comparison of FGMRES  and TSIRM for ex20 of PETSc/SNES with a Block Jacobi  preconditioner  having 100,000 components per core on Curie ($\epsilon_{tsirm}=1e-10$, $max\_iter_{kryl}=30$, $s=12$, $max\_iter_{ls}=15$, $\epsilon_{ls}=1e-40$),  time is expressed in seconds.}
-\label{tab:07}
+\label{tab:08}
 \end{center}
 \end{table*}
 
-\begin{table*}[htbp]
-\begin{center}
-\begin{tabular}{|r|r|r|r|r|r|} 
-\hline
 
-  nb. cores   & \multicolumn{2}{c|}{FGMRES/BJAC} & \multicolumn{2}{c|}{TSIRM CGLS/BJAC} & gain  \\ 
-\cline{2-5}
-                    & Time         & \# Iter.  & Time   & \# Iter. &  \\\hline \hline
-   1024              & 159.52      & 11,584    &  26.34  &     1,563  &  6.06  \\
-   2048             & 226.24       & 16,459    &  37.23 &     2,248   &  6.08\\
-   4096             & 391.21     & 27,794   &  50.93 &   2,911  &  7.69\\
-   8192             & 543.23     & 37,770   &  79.21  &  4,324  & 6.86 \\
 
-\hline
 
-\end{tabular}
-\caption{Comparison of FGMRES  and TSIRM for ex14 of PETSc/SNES with a Block Jacobi  preconditioner  having 100,000 components per core on Curie ($\epsilon_{tsirm}=1e-10$, $max\_iter_{kryl}=30$, $s=12$, $max\_iter_{ls}=15$, $\epsilon_{ls}=1e-40$),  time is expressed in seconds.}
-\label{tab:08}
-\end{center}
-\end{table*}
+\subsection{Influence of parameters for TSIRM}
+
+
 
 
-\subsection{Influcence of parameters for TSIRM}
+
+\subsection{Experiments conclusions }
+
+{\bf A refaire}
+
+Concerning the  experiments some  other remarks are  interesting.
+\begin{itemize}
+\item We have tested other examples  of PETSc/KSP (ex29, ex45, ex49).  For all these
+  examples,  we have also  obtained similar  gains between  GMRES and  TSIRM but
+  those  examples are  not scalable  with many  cores. In  general, we  had some
+  problems with more than $4,096$ cores.
+\item We have tested many iterative  solvers available in PETSc.  In fact, it is
+  possible to use most of them with TSIRM. From our point of view, the condition
+  to  use  a  solver inside  TSIRM  is  that  the  solver  must have  a  restart
+  feature. More precisely,  the solver must support to  be stopped and restarted
+  without decreasing its convergence. That is  why with GMRES we stop it when it
+  is  naturally restarted (\emph{i.e.}   with $m$  the restart  parameter).  The
+  Conjugate Gradient (CG) and all its variants do not have ``restarted'' version
+  in PETSc,  so they are not efficient.   They will converge with  TSIRM but not
+  quickly because  if we  compare a  normal CG with  a CG  which is  stopped and
+  restarted every  16 iterations (for example),  the normal CG will  be far more
+  efficient.   Some  restarted  CG or  CG  variant  versions  exist and  may  be
+  interesting to study in future works.
+\end{itemize}
 
 %%ENDNEW
 
+
 %%%*********************************************************
 %%%*********************************************************
 \section{Conclusion}
