X-Git-Url: https://bilbo.iut-bm.univ-fcomte.fr/and/gitweb/GMRES2stage.git/blobdiff_plain/cce18db44812b9a67dd4d30ca37ea74fd46f16b1..091a1e083b6822c080f3c7524f05e0ee3a87cad4:/paper.tex

diff --git a/paper.tex b/paper.tex
index cf61a9e..e4d146a 100644
--- a/paper.tex
+++ b/paper.tex
@@ -601,13 +601,37 @@ is summarized while intended perspectives are provided.
 %%%*********************************************************
 \section{Related works}
 \label{sec:02} 
-%GMRES method is one of the most widely used iterative solvers chosen to deal with the sparsity and the large order of linear systems. It was initially developed by Saad \& al.~\cite{Saad86} to deal with non-symmetric and non-Hermitian problems, and indefinite symmetric problems too. The convergence of the restarted GMRES with preconditioning is faster and more stable than those of some other iterative solvers. 
+Krylov subspace iteration methods have increasingly become useful and successful
+techniques  for  solving  linear,  nonlinear systems  and  eigenvalue  problems,
+especially      since       the      increase      development       of      the
+preconditioners~\cite{Saad2003,Meijerink77}.  One reason  of  the popularity  of
+these methods is their generality, simplicity and efficiency to solve systems of
+equations arising from very large and complex problems.
+
+GMRES is one of the most  widely used Krylov iterative method for solving sparse
+and large  linear systems. It  is developed by  Saad and al.~\cite{Saad86}  as a
+generalized  method to  deal with  unsymmetric and  non-Hermitian  problems, and
+indefinite symmetric problems too. In its original version called full GMRES, it
+minimizes the residual over the  current Krylov subspace until convergence in at
+most $n$ iterations,  where $n$ is the  size of the sparse matrix.  It should be
+noticed that full GMRES is too expensive in the case of large matrices since the
+required orthogonalization  process per  iteration grows quadratically  with the
+number of iterations. For that reason, in practice GMRES is restarted after each
+$m\ll n$ iterations to avoid the  storage of a large orthonormal basis. However,
+the  convergence behavior  of the  restarted GMRES,  called GMRES($m$),  in many
+cases depends quite critically on  the value of $m$~\cite{Huang89}. Therefore in
+most cases, a preconditioning technique is applied to the restarted GMRES method
+in order to improve its convergence.
+
+In order to enhance the robustness of Krylov iterative solvers, some techniques have been proposed allowing the use of different preconditioners, if necessary, within the iteration instead of restarting. Those techniques may lead to considerable savings in CPU time and memory requirements. Van der Vorst in~\cite{Vorst94} has proposed variants of the GMRES algorithm in which a different preconditioner is applied in each iteration, so-called GMRESR family of nested methods. In fact, the GMRES method is effectively preconditioned with other iterative schemes (or GMRES itself), where the iterations of the GMRES method are called outer iterations while the iterations of the preconditioning process referred to as inner iterations. Saad in~\cite{Saad:1993} has proposed FGMRES which is another variant of the GMRES algorithm using a variable preconditioner. In FGMRES the search directions are preconditioned whereas in GMRESR the residuals are preconditioned. However in practice the good preconditioners are those based on direct methods, as ILU preconditioners, which are not easy to parallelize and suffer from the scalability problems on large clusters of thousands of cores.  
+
+Recently, communication-avoiding methods have been developed to reduce the communication overheads in Krylov subspace iterative solvers. On modern computer architectures, communications between processors are much slower than floating-point arithmetic operations on a given processor. Communication-avoiding techniques reduce either communications between processors or data movements between levels of the memory hierarchy, by reformulating the communication-bound kernels (more frequently SpMV kernels) and the orthogonalization operations within the Krylov iterative solver. Different works have studied the communication-avoiding techniques for the GMRES method, so-called CA-GMRES, on multicore processors and multi-GPU machines~\cite{Mohiyuddin2009,Hoemmen2010,Yamazaki2014}. 
+
+Compared  to all these  works and  to all  the other  works on  Krylov iterative
+method, the originality of our work is to build a second iteration over a Krylov
+iterative method and to minimize the residuals with a least-squares method after
+a given number of outer iterations.
 
-%The next two chapters explore a few methods which are considered currently to be among the most important iterative techniques available for solving large linear systems. These techniques are based on projection processes, both orthogonal and oblique, onto Krylov subspaces, which are subspaces spanned by vectors of the form p(A)v where p is a polynomial. In short, these techniques approximate A â1 b by p(A)b, where p is a âgoodâ polynomial. This chapter covers methods derived from, or related to, the Arnoldi orthogonalization. The next chapter covers methods based on Lanczos biorthogonalization.
-
-%Krylov subspace techniques have inceasingly been viewed as general purpose iterative methods, especially since the popularization of the preconditioning techniqes.
-
-%Preconditioned Krylov-subspace iterations are a key ingredient in many modern linear solvers, including in solvers that employ support preconditioners. 
 %%%*********************************************************
 %%%*********************************************************
 
@@ -615,7 +639,7 @@ is summarized while intended perspectives are provided.
 
 %%%*********************************************************
 %%%*********************************************************
-\section{Two-stage iteration with least-squares residuals minimization algorithm}
+\section{TSIRM: Two-stage iteration with least-squares residuals minimization algorithm}
 \label{sec:03}
 A two-stage algorithm is proposed  to solve large  sparse linear systems  of the
 form  $Ax=b$,  where  $A\in\mathbb{R}^{n\times   n}$  is  a  sparse  and  square
@@ -867,16 +891,15 @@ Core(TM) i7-3630QM CPU @ 2.40GHz with the version 3.5.1 of PETSc.
 
 
 In  Table~\ref{tab:02}, some  experiments comparing  the solving  of  the linear
-systems obtained with the previous matrices  with a GMRES variant and with out 2
-stage algorithm are  given. In the second column, it can  be noticed that either
-GRMES or  FGMRES (Flexible  GMRES)~\cite{Saad:1993} is used  to solve  the linear
-system.   According to  the matrices,  different preconditioner  is  used.  With
-TSIRM, the same solver and the  same preconditionner are used.  This Table shows
-that  TSIRM  can  drastically reduce  the  number  of  iterations to  reach  the
-convergence when the  number of iterations for the normal GMRES  is more or less
-greater than  500. In fact  this also depends  on tow parameters: the  number of
-iterations  to  stop  GMRES  and   the  number  of  iterations  to  perform  the
-minimization.
+systems obtained with the previous matrices  with a GMRES variant and with TSIRM
+are given. In the  second column, it can be noticed that  either GRMES or FGMRES
+(Flexible GMRES)~\cite{Saad:1993} is used to solve the linear system.  According
+to the matrices, different preconditioner  is used.  With TSIRM, the same solver
+and  the  same  preconditionner are  used.   This  Table  shows that  TSIRM  can
+drastically reduce  the number of iterations  to reach the  convergence when the
+number of iterations for  the normal GMRES is more or less  greater than 500. In
+fact this also depends on tow parameters: the number of iterations to stop GMRES
+and the number of iterations to perform the minimization.
 
 
 \begin{table}[htbp]
@@ -967,7 +990,7 @@ preconditioner in PETSc please consult~\cite{petsc-web-page}.
 \hline
 
 \end{tabular}
-\caption{Comparison of FGMRES and TSIRM with FGMRES for example ex15 of PETSc with two preconditioners (mg and sor) with 25,000 components per core on Juqueen (threshold 1e-3, restart=30, s=12),  time is expressed in seconds.}
+\caption{Comparison of FGMRES and TSIRM with FGMRES for example ex15 of PETSc with two preconditioners (mg and sor) with 25,000 components per core on Juqueen ($\epsilon_{tsirm}=1e-3$, $max\_iter_{kryl}=30$, $s=12$, $max\_iter_{ls}=15$, $\epsilon_{ls}=1e-40$),  time is expressed in seconds.}
 \label{tab:03}
 \end{center}
 \end{table*}
@@ -1022,7 +1045,7 @@ the number of iterations. So, the overall benefit of using TSIRM is interesting.
 \begin{tabular}{|r|r|r|r|r|r|r|r|r|} 
 \hline
 
-  nb. cores & threshold   & \multicolumn{2}{c|}{FGMRES} & \multicolumn{2}{c|}{TSIRM CGLS} &  \multicolumn{2}{c|}{TSIRM LSQR} & best gain \\ 
+  nb. cores & $\epsilon_{tsirm}$  & \multicolumn{2}{c|}{FGMRES} & \multicolumn{2}{c|}{TSIRM CGLS} &  \multicolumn{2}{c|}{TSIRM LSQR} & best gain \\ 
 \cline{3-8}
              &                       & Time  & \# Iter.  & Time  & \# Iter. & Time  & \# Iter. & \\\hline \hline
   2,048      & 8e-5                  & 108.88 & 16,560  & 23.06  &  3,630  & 22.79  & 3,630   & 4.77 \\
@@ -1035,7 +1058,7 @@ the number of iterations. So, the overall benefit of using TSIRM is interesting.
 \hline
 
 \end{tabular}
-\caption{Comparison of FGMRES  and TSIRM with FGMRES algorithms for ex54 of Petsc (both with the MG preconditioner) with 25,000 components per core on Curie (restart=30, s=12),  time is expressed in seconds.}
+\caption{Comparison of FGMRES  and TSIRM with FGMRES algorithms for ex54 of Petsc (both with the MG preconditioner) with 25,000 components per core on Curie ($max\_iter_{kryl}=30$, $s=12$, $max\_iter_{ls}=15$, $\epsilon_{ls}=1e-40$),  time is expressed in seconds.}
 \label{tab:04}
 \end{center}
 \end{table*}
@@ -1093,7 +1116,7 @@ taken into account with TSIRM.
 \hline
 
 \end{tabular}
-\caption{Comparison of FGMRES  and TSIRM with FGMRES for ex54 of Petsc (both with the MG preconditioner) with 204,919,225 components on Curie with different number of cores (restart=30, s=12, threshold 5e-5),  time is expressed in seconds.}
+\caption{Comparison of FGMRES  and TSIRM with FGMRES for ex54 of Petsc (both with the MG preconditioner) with 204,919,225 components on Curie with different number of cores ($\epsilon_{tsirm}=5e-5$, $max\_iter_{kryl}=30$, $s=12$, $max\_iter_{ls}=15$, $\epsilon_{ls}=1e-40$),  time is expressed in seconds.}
 \label{tab:05}
 \end{center}
 \end{table*}
