12-10-2014 04

[GMRES2stage.git] / paper.tex

diff --git a/paper.tex b/paper.tex
index 2583813c34ab16744115768e35e4c8d89fca5524..51781ba01ad4fd0363f031d16e054de8d9a270f3 100644 (file)
--- a/paper.tex
+++ b/paper.tex
@@ -601,7 +601,12 @@ is summarized while intended perspectives are provided.
 %%%*********************************************************
 \section{Related works}
 \label{sec:02} 
 %%%*********************************************************
 \section{Related works}
 \label{sec:02} 

-%Wherever Times is specified, Times Roman or Times New Roman may be used. If neither is available on your system, please use the font closest in appearance to Times. Avoid using bit-mapped fonts if possible. True-Type 1 or Open Type fonts are preferred. Please embed symbol fonts, as well, for math, etc.
+Krylov subspace iteration methods have increasingly become useful and successful techniques for solving linear and nonlinear systems and eigenvalue problems, especially since the increase development of the preconditioners~\cite{Saad2003,Meijerink77}. One reason of the popularity of these methods is their generality, simplicity and efficiency to solve systems of equations arising from very large and complex problems. %A Krylov method is based on a projection process onto a Krylov subspace spanned by vectors and it forms a sequence of approximations by minimizing the residual over the subspace formed~\cite{}.
+
+GMRES is one of the most widely used Krylov iterative method for solving sparse and large linear systems. It is developed by Saad and al.~\cite{Saad86} as a generalized method to deal with unsymmetric and non-Hermitian problems, and indefinite symmetric problems too. In its original version called full GMRES, it minimizes the residual over the current Krylov subspace until convergence in at most $n$ iterations, where $n$ is the size of the sparse matrix. It should be noted that full GMRES is too expensive in the case of large matrices since the required orthogonalization process per iteration grows quadratically with the number of iterations. For that reason, in practice GMRES is restarted after each $m\ll n$ iterations to avoid the storage of a large orthonormal basis. However, the convergence behavior of the restarted GMRES in many cases depends quite critically on the value of $m$~\cite{Huang89}. Therefore in most cases, a preconditioning technique is applied to the restarted GMRES method in order to improve its convergence.
+
+%FGMRES , GMRESR, two-stage, communication avoiding 
+

 %%%*********************************************************
 %%%*********************************************************
 
 %%%*********************************************************
 %%%*********************************************************
 


@@ -654,10 +659,10 @@ appropriate than a single direct method in a parallel context.
   \Input $A$ (sparse matrix), $b$ (right-hand side)
   \Output $x$ (solution vector)\vspace{0.2cm}
   \State Set the initial guess $x_0$
   \Input $A$ (sparse matrix), $b$ (right-hand side)
   \Output $x$ (solution vector)\vspace{0.2cm}
   \State Set the initial guess $x_0$

-  \For {$k=1,2,3,\ldots$ until convergence (error$<\epsilon_{tsirm}$)} \label{algo:conv}
+  \For {$k=1,2,3,\ldots$ until convergence ($error<\epsilon_{tsirm}$)} \label{algo:conv}

     \State  $[x_k,error]=Solve(A,b,x_{k-1},max\_iter_{kryl})$   \label{algo:solve}
     \State  $[x_k,error]=Solve(A,b,x_{k-1},max\_iter_{kryl})$   \label{algo:solve}

-    \State $S_{k \mod s}=x_k$ \label{algo:store} \Comment{update column (k mod s) of S}
-    \If {$k \mod s=0$ {\bf and} error$>\epsilon_{kryl}$}
+    \State $S_{k \mod s}=x_k$ \label{algo:store} \Comment{update column ($k \mod s$) of $S$}
+    \If {$k \mod s=0$ {\bf and} $error>\epsilon_{kryl}$}

       \State $R=AS$ \Comment{compute dense matrix} \label{algo:matrix_mul}
             \State $\alpha=Least\_Squares(R,b,max\_iter_{ls})$ \label{algo:}
       \State $x_k=S\alpha$  \Comment{compute new solution}
       \State $R=AS$ \Comment{compute dense matrix} \label{algo:matrix_mul}
             \State $\alpha=Least\_Squares(R,b,max\_iter_{ls})$ \label{algo:}
       \State $x_k=S\alpha$  \Comment{compute new solution}


@@ -675,10 +680,10 @@ method. Moreover,  a tolerance  threshold must be  specified for the  solver. In
 practice, this threshold must be  much smaller than the convergence threshold of
 the  TSIRM algorithm  (\emph{i.e.}, $\epsilon_{tsirm}$).  We also  consider that
 after the call of the $Solve$ function, we obtain the vector $x_k$ and the error
 practice, this threshold must be  much smaller than the convergence threshold of
 the  TSIRM algorithm  (\emph{i.e.}, $\epsilon_{tsirm}$).  We also  consider that
 after the call of the $Solve$ function, we obtain the vector $x_k$ and the error

-which is defined by $||Ax^k-b||_2$.
+which is defined by $||Ax_k-b||_2$.

 
   Line~\ref{algo:store},
 
   Line~\ref{algo:store},

-$S_{k \mod  s}=x^k$ consists in  copying the solution  $x_k$ into the  column $k
+$S_{k \mod  s}=x_k$ consists in  copying the solution  $x_k$ into the  column $k

 \mod s$ of $S$.   After the minimization, the matrix $S$ is  reused with the new
 values of the residuals.  To solve the minimization problem, an iterative method
 is used. Two parameters are required  for that: the maximum number of iterations
 \mod s$ of $S$.   After the minimization, the matrix $S$ is  reused with the new
 values of the residuals.  To solve the minimization problem, an iterative method
 is used. Two parameters are required  for that: the maximum number of iterations


@@ -1100,11 +1105,25 @@ taken into account with TSIRM.
 \end{figure}
 
 
 \end{figure}
 
 

-Concerning the  experiments some  other remarks are  interesting. We  can tested
-other examples  of PETSc  (ex29, ex45,  ex49). For all  these examples,  we also
-obtained  similar  gain between  GMRES  and TSIRM  but  those  examples are  not
-scalable  with many  cores. In  general,  we had  some problems  with more  than
-$4,096$ cores. 
+Concerning the  experiments some  other remarks are  interesting.
+\begin{itemize}
+\item We  can tested other examples of  PETSc (ex29, ex45, ex49).  For all these
+  examples,  we also obtained  similar gain  between GMRES  and TSIRM  but those
+  examples are  not scalable with many  cores. In general, we  had some problems
+  with more than $4,096$ cores.
+\item We have tested many iterative  solvers available in PETSc.  In fast, it is
+  possible to use most of them with TSIRM. From our point of view, the condition
+  to  use  a  solver inside  TSIRM  is  that  the  solver  must have  a  restart
+  feature. More  precisely, the solver must  support to be  stoped and restarted
+  without decrease its  converge. That is why  with GMRES we stop it  when it is
+  naturraly  restarted (i.e.  with  $m$ the  restart parameter).   The Conjugate
+  Gradient (CG) and all its variants do not have ``restarted'' version in PETSc,
+  so they  are not  efficient.  They  will converge with  TSIRM but  not quickly
+  because if  we compare  a normal CG  with a CG  for which  we stop it  each 16
+  iterations  for example,  the  normal CG  will  be for  more efficient.   Some
+  restarted CG  or CG variant versions exist  and may be interested  to study in
+  future works.
+\end{itemize}

 %%%*********************************************************
 %%%*********************************************************
 
 %%%*********************************************************
 %%%*********************************************************