update time table

[chloroplast13.git] / annotated.tex

diff --git a/annotated.tex b/annotated.tex
index 3a7fbd18453a2ccf45285af1b49d51ee931de624..ad0f65412276c72dc410b7d2a565acd6e761608f 100644 (file)
--- a/annotated.tex
+++ b/annotated.tex
@@ -1,65 +1,186 @@
-The  field of genome  annotation pays  a lot  of attentions  where the
-ability  to collect  and analysis  genomical data  can  provide strong
-indicators  for  the study  of  life\cite{Eisen2007}.  Four of  genome
-annotation   centers   (such  as,   \textit{NCBI\cite{Sayers01012011},
+
+These  last years  the cost  of  sequencing genomes  has been  greatly
+reduced,  and thus  more and  more genomes  are  sequenced.  Therefore
+automatic annotation tools are required to deal with this continuously
+increasing amount of genomical data. Moreover, a reliable and accurate
+genome  annotation  process  is  needed  in order  to  provide  strong
+indicators for the study of life\cite{Eisen2007}.
+
+Various  annotation   tools  (\emph{i.e.},  cost-effective  sequencing
+methods\cite{Bakke2009}) producing genomic  annotations at many levels
+of detail  have been designed  by different annotation  centers. Among
+the major annotation  centers we can notice NCBI\cite{Sayers01012011},

 Dogma       \cite{RDogma},       cpBase      \cite{de2002comparative},
 Dogma       \cite{RDogma},       cpBase      \cite{de2002comparative},

-CpGAVAS    \cite{liu2012cpgavas},   and   CEGMA\cite{parra2007cegma}})
-present various types  of annotation tools (\emph{i.e.} cost-effective
-sequencing    methods\cite{Bakke2009})    on   different    annotation
-levels. Generally, previous studies used one of three methods for gene
-finding       in        annotated       genome       using       these
-centers: \textit{alignment-based, composition based, or combination of
-both\cite{parra2007cegma}}. The alignment-based method is used when we
-try  to  predict  a  coding  gene  (\emph{i.e.}.  genes  that  produce
-proteins)  by aligning DNA  sequence of  gene to  the protein  of cDNA
-sequence of homology\cite{parra2007cegma}.  This approach also is used
-in GeneWise\cite{birney2004genewise}.  Composition-based method (known
+CpGAVAS                   \cite{liu2012cpgavas},                   and
+CEGMA\cite{parra2007cegma}. Usually, previous  studies used one out of
+three methods  for finding  genes in annoted  genomes using  data from
+these  centers: \textit{alignment-based},  \textit{composition based},
+or a  combination of both~\cite{parra2007cegma}.   The alignment-based
+method  is used  when trying  to predict  a coding  gene (\emph{i.e.}.
+genes that produce proteins) by aligning a genomic DNA sequence with a
+cDNA  sequence  coding  an homologous  protein  \cite{parra2007cegma}.
+This approach is  also used in GeneWise\cite{birney2004genewise}.  The
+alternative   method,   the    composition-based   one   (also   known

 as  \textit{ab initio})  is based  on  a probabilistic  model of  gene
 structure  to  find genes  according  to  the  gene value  probability
 as  \textit{ab initio})  is based  on  a probabilistic  model of  gene
 structure  to  find genes  according  to  the  gene value  probability

-(GeneID\cite{parra2000geneid}).  In this  section, we  consider  a new
-method of finding core genes from large amount of chloroplast genomes,
-as  a solution  of the  problem resulting  from the  method  stated in
-section  two. This  method is  based  on extracting  gene features.  A
-general overview of the system is illustrated in Figure \ref{Fig1}.\\
-
-\begin{figure}[H]  
+(GeneID \cite{parra2000geneid}).  Such  annotated genomic data will be
+used to overcome  the limitation of the first  method described in the
+previous section.   In fact, the  second method we propose  finds core
+genes  from  large  amount  of  chloroplast  genomes  through  genomic
+features extraction.
+
+Figure~\ref{Fig1} presents an overview  of the entire method pipeline.
+More    precisely,    the   second    method    consists   of    three
+stages:   \textit{Genome    annotation},   \textit{Core   extraction},
+and    \textit{Features    Visualization}    which   highlights    the
+relationships.  To  understand the  whole core extraction  process, we
+describe briefly each  stage below. More details will  be given in the
+coming subsections.   The method uses as starting  point some sequence
+database  chosen  among   the  many  international  databases  storing
+nucleotide sequences, like  the GenBank at NBCI \cite{Sayers01012011},
+the    \textit{EMBL-Bank}     \cite{apweiler1985swiss}    in    Europe
+or   \textit{DDBJ}   \cite{sugawara2008ddbj}   in  Japan.    Different
+biological tools can analyze  and annotate genomes by interacting with
+these databases to  align and extract sequences to  predict genes. The
+database in  our method must be  taken from any  confident data source
+that stores annotated and/or unannotated chloroplast genomes.  We have
+considered the GenBank-NCBI \cite{Sayers01012011} database as sequence
+database:  99~genomes of chloroplasts  were retrieved.   These genomes
+lie in  the eleven type  of chloroplast families and  Table \ref{Tab2}
+summarizes their distribution in our dataset.
+
+\begin{figure}[h]  

   \centering
   \centering

-    \includegraphics[width=0.7\textwidth]{generalView}
-\caption{A general overview of the system}\label{Fig1}
+    \includegraphics[width=0.75\textwidth]{generalView}
+\caption{A general overview of the annotation-based approach}\label{Fig1}

 \end{figure}
 
 \end{figure}
 

-In Figure 1, we illustrate the general overview of system pipeline: \textit{Database, Genomes annotation, Core extraction,} and \textit{relationships}. We will give a short discussion for each stage of the model in order to understand the whole core extraction process. This work starts with a gene Bank database; however, many international Banks for nucleotide sequence databases (such as, \textit{GenBank} \cite{Sayers01012011} in USA, \textit{EMBL-Bank} \cite{apweiler1985swiss} in Europe, and \textit{DDBJ} \cite{sugawara2008ddbj} in Japon) exist to store various genomes and DNA species. Different biological tools can analyse and annotate genomes by interacting with these databases to  align and extract sequences to predict genes. The database in this model must be taken from any confident data source that stores annotated and/or unannotated chloroplast genomes. We consider GenBank-NCBI \cite{Sayers01012011} database to be our nucleotide sequences database. Annotation (as the second stage) is considered to be the first important task for extract gene features. Good annotation tool leads us to extract good gene feature. In this paper, two annotation techniques from \textit{NCBI, and Dogma} are used to extract \textit{genes features}. Extracting gene feature (as a third stage) can be anything like (genes names, gene sequences, protein sequence,...etc). Our methodology in this paper consider gene names, genes counts, and gene sequence for extracting core genes and producing chloroplast evolutionary tree. \\
-In last stage, features visualization represents methods to visualize genomes and/or gene evolution in chloroplast. We use the forms of tables, phylogenetic trees, graphs,...,etc to organize and represent genomes relationships to achieve the goal of representing gene evolution. In addition, comparing these forms with another annotation tool forms dedicated to large population of chloroplast genomes give us biological perspectives to the nature of chloroplasts evolution. \\
-A local database attached with each pipe stage is used to store all the informations of extraction process. The output from each stage in our system will be an input to the second stage and so on.
+Annotation,  which  is the  first  stage,  is  an important  task  for
+extracting gene features. Indeed, to extract good gene feature, a good
+annotation tool  is obviously  required. To obtain  relevant annotated
+genomes, two annotation  techniques from NCBI and Dogma  are used. The
+extraction of gene feature, the  next stage, can be anything like gene
+names,  gene  sequences, protein  sequences,  and  so  on. Our  method
+considers gene  names, gene counts,  and gene sequence  for extracting
+core  genes and  producing  chloroplast evolutionary  tree. The  final
+stage   allows  to   visualize  genomes   and/or  gene   evolution  in
+chloroplast.    Therefore   we   use  representations   like   tables,
+phylogenetic  trees,  graphs,  etc.   to  organize  and  show  genomes
+relationships,  and  thus  achieve   the  goal  of  representing  gene
+evolution.   In addition,  comparing these  representations  with ones
+issued from  another annotation tool dedicated to  large population of
+chloroplast genomes  give us biological perspectives to  the nature of
+chloroplasts evolution. Notice that  a local database linked with each
+pipe stage is  used to store all the  informations produced during the
+process.

 
 

-\subsection{Genomes Samples}
-In this research, we retrieve genomes of Chloroplasts from NCBI. Ninety nine genome of them are considered to work with. These genomes lies in the eleven type of chloroplast families. The distribution of genomes is illustrated in detail in Table \ref{Tab2}.
-       

 \input{population_Table}
 \input{population_Table}

-\subsection{Genome Annotation Techniques}
-Genome annotation is the second stage in the model pipeline. Many techniques were developed to annotate chloroplast genomes but the problem is that they vary in the number and type of predicted genes (\emph{i.e.} the ability to predict genes and \textit{for example: Transfer RNA (tRNA)} and \textit{Ribosomal RNA (rRNA)} genes). Two annotation techniques from NCBI and Dogma are considered to analyse chloroplast genomes to examine the accuracy of predicted coding genes.   
+       
+\subsection{Genome annotation techniques}
+
+For  the first  stage, genome  annotation, many  techniques  have been
+developed  to annotate chloroplast  genomes.  These  techniques differ
+from  each others  in  the number  and  type of  predicted genes  (for
+example:  \textit{Transfer  RNA   (tRNA)}  and  \textit{Ribosomal  RNA
+(rRNA)}  genes). Two  annotation techniques  from NCBI  and  Dogma are
+considered to analyze chloroplast genomes.

 
 \subsubsection{Genome annotation from NCBI} 
 
 \subsubsection{Genome annotation from NCBI} 

-The objective from this step is to organize genes, solve gene duplications, and generate sets of genes from each genome. The input to the system is a list of chloroplast genomes, annotated from NCBI. All genomes stored as \textit{.fasta} files which have a collection of protein coding genes\cite{parra2007cegma,RDogma} (gene that produce proteins) with its coding sequences.
-As a preprocessing step to build the set of core genes, we need to analyse these genomes (using \textit{BioPython} package\cite{chapman2000biopython}). The process starts by converting each genome from fasta format to GenVision\cite{geneVision} format from DNASTAR. The outputs from this operation are lists of genes for each genome, their gene names and gene counts. In this stage, we accumulate some gene duplications for each treated genome. These gene name duplication can come from gene fragments, (e.g. gene fragments treated with NCBI), and from chloroplast DNA sequences. To ensure that all the duplications are removed, each list of genes is translated into a set of genes. NCBI genome annotation produce genes except \textit{Ribosomal rRNA}.

 
 

-\subsubsection{Genome annotation from Dogma}
-Dogma is an annotation tool developed in the university of Texas in 2004. Dogma is an abbreviation of (\textit{Dual Organellar GenoMe Annotator}) for plant chloroplast and animal mitochondrial genomes.
-It has its own database for translating the genome in all six reading frames and it queries the amino acid sequence database using Blast\cite{altschul1990basic}(\emph{i.e.} Blastx) with various parameters. Furthermore, identify protein coding genes in the input genome based on sequence similarity of genes in Dogma database. In addition, it can produce the \textit{Transfer RNAs (tRNA)}, and the \textit{Ribosomal RNAs (rRNA)} and verifies their start and end positions rather than NCBI annotation tool. There is no gene duplication with dogma after solving gene fragmentation. \\
-Genome annotation with dogma can be the key difference of extracting core genes. The step of annotation is divided into two tasks: first, It starts to annotate complete chloroplast genomes (\emph{i.e.} \textit{Unannotate genome from NCBI} by using Dogma web tool. This process is done manually. The output from dogma is considered to be a collection of coding genes files for each genome in the form of GeneVision file format. 
-The second task is to solve gene fragments. Two methods are used to solve gene duplication. First, for the method based on gene name, all the duplications are removed, where each list of genes is translated into a set of genes. Second, for the method of gene quality test, a defragment process used to avoid gene duplication. \\
-In each iteration, this process starts by taking one gene from gene list, searches for gene duplication, if exists, it looks on the orientation of the fragment sequence: if it is positive, then it appends fragment sequence to a gene files. Otherwise, the process applies reverse complement operations on gene sequences and appends it to gene files. An additional process is then applied to check start and stop codons in case of missing. All genomes after this stage are fully annotated, their genes are de-fragmented, and counts are identified.\\
+The objective  is to generate sets  of genes from each  genome so that
+genes are organized  without any duplication.  The input  is a list of
+chloroplast genomes  annotated from NCBI. More  precisely, all genomes
+are stored as \textit{.fasta} files  which consists in a collection of
+protein  coding genes\cite{parra2007cegma,RDogma}  (gene  that produce
+proteins) organized in coding sequences.   To be able build the set of
+core    genes,     we    need    to     preprocess    these    genomes
+using  \textit{BioPython}  package \cite{chapman2000biopython}.   This
+step  starts by  converting  each  genome from  FASTA  file format  to
+GenVision \cite{geneVision}  format from DNASTAR. Each  genome is thus
+converted in  a list of genes,  with gene names and  gene counts. Gene
+name duplications can be accumulated during the treatment of a genome.
+These  duplications  come   from  gene  fragments  (\emph{e.g.}   gene
+fragments treated  with NCBI) and  from chloroplast DNA  sequences. To
+ensure that  all the  duplications are removed,  each list of  gene is
+translated  into a  set of  genes.  Note that  NCBI genome  annotation
+produces genes except \textit{Ribosomal (rRNA)} genes.

 
 

-\subsection{Core Genes Extraction}
-The goal of this step is to extract maximum core genes from sets of genes. The methodology of finding core genes is as follow: \\
+\subsubsection{Genome annotation from Dogma}

 
 

-\subsubsection{Pre-Processing}
-We apply two pre-processing methods to organize and prepare genomes data: the first method based on gene name and count, and the second one is based on sequence quality control test.\\
-In the first method, we extract a list of genes from each chloroplast genome. Then we store this list of genes in the database under genome name. Genes counts can be extracted by a specific length command. \textit{Intersection Core Matrix}  then applied to extract the core genes. The problem with this method is how can we ensure that the gene which is predicted in core genes is the same gene in leaf genomes? The answer of this question is as follows: if the sequence of any gene in a genome annotated from dogma and NCBI are similar with respect to a threshold, we do not have any problem with this method. Otherwise, we have a problem, because we can not decide which sequence goes to a gene in core genes.
-The second pre-processing method states: we can predict the best annotated genome by merging the annotated genomes from NCBI and dogma if we follow the quality of genes names and sequences test. To generate all quality genes of each genome, the hypothesis state: any gene will be in predicted genome if and only if the annotated genes between NCBI and Dogma pass a specific threshold of \textit{quality control test}. To accept the quality test, we applied Needle-man Wunch algorithm to compare two gene sequences with respect to a threshold. If the alignment score passes the threshold, then the gene will be in the predicted genome. Otherwise, the gene is ignored. After predicting all genomes, \textit{Intersection Core Matrix} is applied on these new genomes to extract core genes, as shown in Algorithm \ref{Alg3:thirdM}.         
+Dogma stands for \textit{Dual  Organellar GenoMe Annotator}.  It is an
+annotation tool  developed at  University of Texas  in 2004  for plant
+chloroplast and  animal mitochondrial genomes.  This tool  has its own
+database  for translating  a  genome  in all  six  reading frames  and
+queries     the     amino     acid     sequence     database     using
+BLAST  \cite{altschul1990basic}  (\emph{i.e.}   Blastx)  with  various
+parameters.  Protein  coding genes are  identified in an  input genome
+using sequence similarity of genes  in Dogma database.  In addition in
+comparison   with   NCBI    annotation   tool,   Dogma   can   produce
+both \textit{Transfer RNAs (tRNA)} and \textit{Ribosomal RNAs (rRNA)},
+verify their start and end  positions. Another difference is also that
+there  is   no  gene  duplication   with  Dogma  after   solving  gene
+fragmentation. In  fact, genome annotation  with Dogma can be  the key
+difference when extracting core genes.
+
+The Dogma  annotation process  is divided into  two tasks.   First, we
+manually annotate chloroplast genomes using Dogma web tool. The output
+of this step is supposed to  be a collection of coding genes files for
+each genome, organized in GeneVision file. The second task is to solve
+the  gene   duplication  problem  and   therefore  we  have   use  two
+methods. The first method, based  on gene name, translates each genome
+into a set  of genes without duplicates. The  second method avoid gene
+duplication  through a  defragment  process. In  each iteration,  this
+process  starts by taking  a gene  from gene  list, searches  for gene
+duplication, if a duplication is found, it looks on the orientation of
+the  fragment sequence.   If it  is positive  it appends  directly the
+sequence to  gene files.  Otherwise reverse  complement operations are
+applied  on the sequence,  which is  then also  append to  gene files.
+Finally, a  check for missing start  and stop codons  is performed. At
+the  end  of  the  annotation  process,  all  the  genomes  are  fully
+annotated,  their   genes  are  defragmented,  and   gene  counts  are
+available.
+
+\subsection{Core genes extraction}
+
+The goal of  this stage is to extract maximum core  genes from sets of
+genes.  To find core genes, the following methodology is applied.
+
+\subsubsection{Preprocessing}
+
+In order  to extract  core genomes in  a suitable manner,  the genomic
+data are preprocessed with two methods: on the one hand a method based
+on gene  name and count,  and on  the other hand  a method based  on a
+sequence quality control test.
+
+In the first method, we extract  a list of genes from each chloroplast
+genome.  Then we store this list of genes in the database under genome
+nam and  genes counts can be  extracted by a  specific length command.
+The \textit{Intersection  Core Matrix}, described  in next subsection,
+is then  computed to  extract the core  genes.  The problem  with this
+method can be stated as follows: how can we ensure that the gene which
+is  predicted in  core genes  is the  same gene  in leaf  genomes? The
+answer  to this problem  is that  if the  sequences of  any gene  in a
+genome annotated  from Dogma  and NCBI are  similar with respect  to a
+given  threshold,  then   we  do  not  have  any   problem  with  this
+method. When the sequences are  not similar we have a problem, because
+we cannot decide which sequence belongs to a gene in core genes.
+
+The second method is based on  the underlying idea: we can predict the
+the best annotated  genome by merging the annotated  genomes from NCBI
+and Dogma according to a quality test on genes names and sequences. To
+obtain all  quality genes  of each genome,  we consider  the following
+hypothesis: any gene  will appear in the predicted  genome if and only
+if the  annotated genes  in NCBI and  Dogma pass a  specific threshold
+of  \textit{quality  control test}.   In  fact,  the Needle-man  Wunch
+algorithm  is applied  to compare  both  sequences with  respect to  a
+threshold. If  the alignment  score is above  the threshold,  then the
+gene will be  retained in the predicted genome,  otherwise the gene is
+ignored.   Once    the   prediction   of   all    genomes   is   done,
+the \textit{Intersection Core Matrix} is computed on these new genomes
+to extract core genes, as explained in Algorithm \ref{Alg3:thirdM}.

 
 \begin{algorithm}[H]
 
 \begin{algorithm}[H]

-\caption{Extract new genome based on Gene Quality test}
+\caption{Extract new genome based on gene quality test}

 \label{Alg3:thirdM}
 \begin{algorithmic} 
 \REQUIRE $Gname \leftarrow \text{Genome Name}, Threshold \leftarrow 65$
 \label{Alg3:thirdM}
 \begin{algorithmic} 
 \REQUIRE $Gname \leftarrow \text{Genome Name}, Threshold \leftarrow 65$


@@ -80,131 +201,174 @@ The second pre-processing method states: we can predict the best annotated genom
 \end{algorithmic}
 \end{algorithm}
 
 \end{algorithmic}
 \end{algorithm}
 

-\textbf{geneChk} is a subroutine, it is used to find the best similarity score between two gene sequences after applying operations like \textit{reverse, complement, and reverse complement}. The algorithm of geneChk is illustrated in Algorithm \ref{Alg3:genechk}.
+\textbf{geneChk} is a subroutine used to find the best similarity score between 
+two gene sequences after applying operations like \textit{reverse}, {\it complement}, 
+and {\it reverse complement}. Algorithm~\ref{Alg3:genechk} gives the outline of 
+geneChk subroutine.

 
 \begin{algorithm}[H]
 
 \begin{algorithm}[H]

-\caption{Find the Maximum similarity score between two sequences}
+\caption{Find the Maximum Similarity Score between two sequences}

 \label{Alg3:genechk}
 \begin{algorithmic} 
 \label{Alg3:genechk}
 \begin{algorithmic} 

-\REQUIRE $gen1,gen2 \leftarrow \text{NCBI gene sequence, Dogma gene sequence}$
+\REQUIRE $g1,g2 \leftarrow \text{NCBI gene sequence, Dogma gene sequence}$

 \ENSURE $\text{Maximum similarity score}$
 \ENSURE $\text{Maximum similarity score}$

-\STATE $Score1 \leftarrow needle(gen1,gen2)$
-\STATE $Score2 \leftarrow needle(gen1,Reverse(gen2))$
-\STATE $Score3 \leftarrow needle(gen1,Complement(gen2))$
-\STATE $Score4 \leftarrow needle(gen1,Reverse(Complement(gen2)))$
-\RETURN $max(Score1, Score2, Score3, Score4)$
+\STATE $score1 \leftarrow needle(g1,g2)$
+\STATE $score2 \leftarrow needle(g1,Reverse(g2))$
+\STATE $score3 \leftarrow needle(g1,Complement(g2))$
+\STATE $score4 \leftarrow needle(g1,Reverse(Complement(g2)))$
+\RETURN $max(score1,score2,score3,score4)$

 \end{algorithmic}
 \end{algorithm}  
 
 \subsubsection{Intersection Core Matrix (\textit{ICM})}
 
 \end{algorithmic}
 \end{algorithm}  
 
 \subsubsection{Intersection Core Matrix (\textit{ICM})}
 

-The idea behind extracting core genes is to iteratively collect the maximum number of common genes between two genomes. To do so, the system builds an \textit{Intersection Core Matrix (ICM)}. ICM is a two dimensional symmetric matrix where each row and each column represents one genome. Each position in ICM stores the \textit{Intersection Scores(IS)}. IS is the cardinality number of a core genes which comes from intersecting one genome with other ones. Maximum cardinality results to select two genomes with their maximum core. Mathematically speaking, if we have an $n \times n$ matrix where $n$  
-is the number of genomes in local database, then lets consider:\\
-
+To extract  core genes, we  iteratively collect the maximum  number of
+common  genes   between  genomes  and  therefore   during  this  stage
+an \textit{Intersection  Core Matrix}  (ICM) is built.   ICM is  a two
+dimensional symmetric matrix where each row and each column correspond
+to   one   genome.   Hence,   an   element   of   the  matrix   stores
+the  \textit{Intersection Score}  (IS):  the cardinality  of the  core
+genes   set  obtained   by  intersecting   one  genome   with  another
+one. Maximum  cardinality results in selecting the  two genomes having
+the maximum score. Mathematically speaking, if we have $n$ genomes in
+local database, the ICM is an $n \times n$ matrix whose elements
+satisfy: 

 \begin{equation}
 \begin{equation}

-Score=\max_{i<j}\vert x_i \cap x_j\vert
+score_{ij}=\vert g_i \cap g_j\vert

 \label{Eq1}
 \end{equation}
 \label{Eq1}
 \end{equation}

-
-\noindent where $x_i, x_j$ are elements in the matrix. The generation of a new core genes is depending on the cardinality value of intersection scores, we call it \textit{Score}:
-$$\text{New Core} = \begin{cases} 
-\text{Ignored} & \text{if $\textit{Score}=0$;} \\
-\text{new Core id} & \text{if $\textit{Score}>0$.}
-\end{cases}$$
-
-if $\textit{Score}=0$ then we have \textit{disjoint relation} \emph{i.e.}, no common genes between two genomes. In this case the system ignores the genome that annul the core gene size. Otherwise, The system removes these two genomes from ICM and add new core genome with a \textit{coreID} of them to ICM for the calculation in next iteration. This process reduces the size of ICM and repeats until all genomes are treated \emph{i.e.} ICM has no more genomes.   
-We observe that ICM is very large because of the amount of data that it stores. This results to be time and memory consuming for calculating the intersection scores. To increase the speed of calculations, it is sufficient to only calculate the upper triangle scores. The time complexity for this process after enhancement is thus $O(\frac{n.(n-1)}{2})$. Algorithm \ref{Alg1:ICM} illustrates the construction of the ICM matrix and the extraction of the core genes where \textit{GenomeList}, represents the database where all genomes data are stored. At each iteration, it computes the maximum core genes with its two genomes parents.
+\noindent where $1 \leq i \leq n$, $1 \leq j \leq n$, and $g_i, g_j$ are 
+genomes. The  generation of a new  core gene depends  obviously on the
+value  of the  intersection scores  $score_{ij}$. More  precisely, the
+idea is  to consider a  pair of genomes  such that their score  is the
+largest element in ICM. These two genomes are then removed from matrix
+and the  resulting new  core genome is  added for the  next iteration.
+The ICM is then updated to take into account the new core gene: new IS
+values are computed for it. This process is repeated until no new core
+gene can be obtained.
+
+We  can observe  that  the ICM  is very  large  due to  the amount  of
+data. As a consequence, the  computation of the intersection scores is
+both  time and  memory consuming.  However,  since ICM  is a  symetric
+matrix we can reduce the  computation overhead by considering only its
+triangular  upper part.  The  time complexity  for this  process after
+enhancement is thus $O(\frac{n.(n-1)}{2})$.  Algorithm ~\ref{Alg1:ICM}
+illustrates the construction  of the ICM matrix and  the extraction of
+the  core  genes, where  \textit{GenomeList}  represents the  database
+storing all genomes  data. At each iteration, it  computes the maximum
+core genes with its two genomes parents.
+
+% ALGORITHM HAS BEEN REWRITTEN

 
 \begin{algorithm}[H]
 \caption{Extract Maximum Intersection Score}
 \label{Alg1:ICM}
 \begin{algorithmic} 
 
 \begin{algorithm}[H]
 \caption{Extract Maximum Intersection Score}
 \label{Alg1:ICM}
 \begin{algorithmic} 

-\REQUIRE $L \leftarrow \text{genomes vectors}$
-\ENSURE $B1 \leftarrow Max core vector$ 
+\REQUIRE $L \leftarrow \text{genomes sets}$
+\ENSURE $B1 \leftarrow \text{Max Core set}$ 

 \FOR{$i \leftarrow 0:len(L)-1$}
 \FOR{$i \leftarrow 0:len(L)-1$}

+        \STATE $score \leftarrow 0$

        \STATE $core1 \leftarrow set(GenomeList[L[i]])$
        \STATE $core1 \leftarrow set(GenomeList[L[i]])$

-       \STATE $score1 \leftarrow 0$
-       \STATE $g1,g2 \leftarrow$ " "
+       \STATE $g1 \leftarrow L[i]$

        \FOR{$j \leftarrow i+1:len(L)$}
        \FOR{$j \leftarrow i+1:len(L)$}

-               \STATE $core2 \leftarrow set(GenomeList[L[i]])$
-               \IF{$i < j$}
-                       \STATE $Core \leftarrow core1 \cap core2$
-                       \IF{$len(Core) > score1$}
-                               \STATE $g1 \leftarrow L[i]$
-                               \STATE $g2 \leftarrow L[j]$
-                               \STATE $Score \leftarrow len(Core)$
-                       \ELSIF{$len(Core) == 0$}
-                               \STATE $g1 \leftarrow L[i]$
-                               \STATE $g2 \leftarrow L[j]$
-                               \STATE $Score \leftarrow -1$
-                       \ENDIF
-               \ENDIF
+               \STATE $core2 \leftarrow set(GenomeList[L[j]])$
+               \STATE $Core \leftarrow core1 \cap core2$
+               \IF{$len(Core) > score$}
+                  \STATE $score \leftarrow len(Core)$
+                 \STATE $g2 \leftarrow L[j]$
+                \ENDIF

        \ENDFOR
        \ENDFOR

-       \STATE $B1[score1] \leftarrow (g1,g2)$
+       \STATE $B1[score] \leftarrow (g1,g2)$

 \ENDFOR
 \RETURN $max(B1)$
 \end{algorithmic}
 \end{algorithm}
 
 \ENDFOR
 \RETURN $max(B1)$
 \end{algorithmic}
 \end{algorithm}
 

-\subsection{Features Visualization}
-The goal here is to visualize results by building a tree of evolution. All core genes generated with their genes are very important information in the tree, because they can be viewed as an ancestor information for two genomes or more. Further more, each node in the tree represents one chloroplast genome or one predicted core which named under the title of \textit{(Genes count:Family name\_Scientific names\_Accession number)}, Edges represent the number of lost genes from each leaf genome or from an intermediate core genes. The number of lost genes here can represent an important factor for evolution: it represents how much is the lost of genes from the species belongs to same or different families. By the principle of classification, a small number of gene lost among species indicates that those species are close to each other and belong to same family, while big genes lost means that we have an evolutionary relationship between species from different families. To see the picture clearly, Phylogenetic tree is an evolutionary tree generated also by the system. Generating this tree is based on the distances among genes sequences. There are many resources to build such tree (for example: PHYML\cite{guindon2005phyml}, RAxML{\cite{stamatakis2008raxml,stamatakis2005raxml}, BioNJ , and TNT\cite{goloboff2008tnt}}. We consider to use RAxML\cite{stamatakis2008raxml,stamatakis2005raxml} because it is fast and accurate for build large trees for large count of genomes sequences. The procedure of constructing phylogenetic tree stated in the following steps:
-
+\subsection{Features visualization}
+
+The goal is to visualize results  by building a tree of evolution. All
+core  genes generated  represent  an important information  in the  tree,
+because they  provide ancestor information of two  or more
+genomes. Each  node in the  tree represents one chloroplast  genome or
+one predicted core and labelled as \textit{(Genes count:Family name\_Scientific
+names\_Accession number)}. While an edge is labelled with the number of
+lost  genes from  a leaf  genome or  an intermediate  core  gene. Such
+numbers are  very interesting because  they give an  information about
+the evolution:  how many genes  were lost between two  species whether
+they  belong  to  the  same  family  or not.   By  the  principle  of
+classification, a  small number of genes lost  among species indicates
+that those species are close to  each other and belong to same family,
+while a  large lost  means that we  have an  evolutionary relationship
+between species  from different families. To depict  the links between
+species   clearly,  we   built   a  phylogenetic   tree  showing   the
+relationships based on the distances among genes sequences. Many tools
+are    available   to    obtain    a   such    tree,   for    example:
+PHYML\cite{guindon2005phyml},
+RAxML{\cite{stamatakis2008raxml,stamatakis2005raxml},    BioNJ,    and
+TNT\cite{goloboff2008tnt}}.    In   this  work,   we   chose  to   use
+RAxML\cite{stamatakis2008raxml,stamatakis2005raxml}   because   it  is
+fast, accurate,  and can build large  trees when dealing  with a large
+number of genomic sequences.
+
+The procedure used to built a phylogenetic tree is as follows:

 \begin{enumerate}
 \begin{enumerate}

-\item Extract gene sequence for all gene in all core genes, store it in database.
-\item Use multiple alignment tool such as (****to be write after see christophe****) to align these sequences with each others.
-\item aligned genomes sequences then submitted to RAxML program to compute the distances and draw phylogenetic tree.
+\item For each gene in a core gene, extract its sequence and store it in the database.
+\item Use multiple alignment tools such as (****to be write after see christophe****) 
+to align these sequences with each others.
+\item we use an outer-group genome from cyanobacteria to calculate distances.
+\item Submit the resulting aligned sequences to RAxML program to compute the distances and finally draw the phylogenetic tree.

 \end{enumerate} 
 
 \begin{figure}[H]
 \end{enumerate} 
 
 \begin{figure}[H]

-  \centering
-    \includegraphics[width=0.7\textwidth]{Whole_system}
-    \caption{Total overview of the system pipeline}\label{wholesystem}
+  \centering \includegraphics[width=0.75\textwidth]{Whole_system}
+  \caption{Overview of the pipeline}\label{wholesystem}

 \end{figure}
 
 \section{Implementation}
 \end{figure}
 
 \section{Implementation}

-We implemented four algorithms to extract maximum core genes from large amount of chloroplast genomes. Two algorithms used to extract core genes based on NCBI annotation, and the others based on dogma annotation tool. Evolutionary tree generated as a result from each method implementation. In this section, we will present the four methods, and how they can extract maximum core genes?, and how the developed code will generate the evolutionary tree.
+We implemented the three algorithms using dell laptop model latitude E6430 with 6 GB of memory, and Intel core i5 processor of 2.5 Ghz$\times 4$ with 3 MB of CPU cash. We built the code using python version 2.7 under ubuntu 12.04 LTS. We also used python packages such as os, Biopython, memory\_profile, re, numpy, time, shutil, and xlsxwriter to extract core genes from large amount of chloroplast genomes. Table \ref{Etime}, show the annotation type, execution time, and the number of core genes for each method:
+
+\begin{center}
+\begin{tiny}
+\begin{table}[H]
+\caption{Type of Annotation, Execution Time, and core genes for each method}\label{Etime}
+\begin{tabular}{p{2.5cm}p{0.5cm}p{0.5cm}p{0.5cm}p{0.5cm}p{0.5cm}p{0.5cm}p{0.5cm}p{0.5cm}p{0.5cm}p{0.2cm}}
+\hline\hline
+ & \multicolumn{2}{c}{Annotation} & \multicolumn{2}{c}{Features} & \multicolumn{2}{c}{E. Time} & \multicolumn{2}{c}{C. genes} & \multicolumn{2}{c}{Bad Gen.} \\
+~ & N & D & Name & Seq & N & D & N & D & N & D \\
+\hline
+Gene prediction & $\surd$ & - & - & $\surd$ & ? & - & ? & - & 0 & -\\[0.5ex]
+Gene Features & $\surd$ & $\surd$ & $\surd$ & - & 4.98 & 1.52 & 28 & 10 & 1 & 0\\[0.5ex]
+Gene Quality & $\surd$ & $\surd$ & $\surd$ & $\surd$ & \multicolumn{2}{c}{$\simeq$3 days + 1.29} & \multicolumn{2}{c}{4} & \multicolumn{2}{c}{1}\\[1ex]
+\hline
+\end{tabular}
+\end{table}
+\end{tiny}
+\end{center} 
+
+In table \ref{Etime}, we show that all methods need low execution time to finish extracting core genes from large chloroplast genomes except in gene quality method where we need about 3-4 days for sequence comparisons to construct quality genomes then it takes just 1.29 minute to extract core genes. This low execution time give us a privilage to use these methods to extract core genes on a personal comuters rather than main frames or parallel computers. In the table, \textbf{N} means NCBI, \textbf{D} means DOGMA, and \textbf{Seq} means Sequence. Annotation is represent the type of algorithm used to annotate chloroplast genome. We can see that the two last methods used the same annotation sources. Features means the type of gene feature used to extract core genes, and this is done by extracting gene name, gene sequence, or both of them. The execution time is represented the whole time needed to extract core genes in minutes. We can see in the table that the second method specially with DOGMA annotation has the lowest execution time of 1.52 minute. In last method We needs approxemetly three days (this period is depend on the amount of genomes) to finish the operation of extracting quality genomes only, while the execution time will be 1.29 minute if we have quality genomes. The number of core genes is represents the amount of genes in the last core genome. The main goal is to find the maximum core genes that simulate biological background of chloroplasts. With NCBI we have 28 genes for 96 genomes instead of 10 genes with DOGMA for 97 genomes. But the biological distribution of genomes with NCBI in core tree did not reflect good biological perspective. While in the core tree with DOGMA, the distribution of genomes are biologically good. Bad genomes are the number of genomes that destroy core genes because of the low number of gene intersection. \textit{NC\_012568.1 Micromonas pusilla}, is the only genome that observed to destroy the core genome with NCBI based on the method of gene features and in the third method of gene quality.  \\
+
+The second important factor is the amount of memory usage in each methodology. Table \ref{mem} show the amounts of memory consumption by each method.
+
+\begin{center}
+\begin{tiny}
+\begin{table}[H]
+\caption{Memory usages in (MB) for each methodology}\label{mem}
+\begin{tabular}{p{2.5cm}p{1.5cm}p{1cm}p{1cm}p{1cm}p{1cm}p{1cm}p{1cm}}
+\hline\hline
+Method& & Load Gen. & Conv. gV & Read gV & ICM & Core tree & Core Seq. \\
+\hline
+Gene prediction & ~ & ~ & ~ & ~ & ~ & ~ & ~\\
+\multirow{2}{*}{Gene Features} & NCBI & 15.4 & 18.9 & 17.5 & 18 & 18 & 28.1\\
+              & DOGMA& 15.3 & 15.3 & 16.8 & 17.8 & 17.9 & 31.2\\
+Gene Quality  & ~ & 15.3 & $\le$3G & 16.1 & 17 & 17.1 & 24.4\\ 
+\hline
+\end{tabular}
+\end{table}
+\end{tiny}
+\end{center}  
+
+We used a package from PyPI~(\textit{the Python Package Index}) where located at~ (https://pypi.python.org/pypi) named \textit{Memory\_profile} to extract all the values in table \ref{mem}. In this table, all the values are presented in mega bytes and \textit{gV} means genevision file format. We see that all memory levels in all methods are reletively low and can be available in any personal computer. All memory values shows that the method of gene features based on DOGMA annotation have the more resonable memory values to extract core genome from loading genomes until extracting core sequences. The third method, gives us the lowest values if we already have the quality genomes, but it will consume high memory locations if we do not have them. Also, the amount of memory locations in the third method vary according to the size of each genome.\\

 
 

-\subsection{Extract Core Genes based on Gene Contents}

 
 

-\subsubsection{Core Genes based on NCBI Annotation}
-The first idea to construct the core genome is based on the extraction of Genes names (as gene presence or absence). For instant, in this stage neither sequence comparison nor new annotation were made, we just want to extract all genes with counts stored in each chloroplast genome, then find the intersection core genes based on gene names. \\
-The pipeline of extracting core genes can summarize in the following steps according to pre-processing method used:\\

 
 

-\begin{enumerate}
-\item We downloads already annotated chloroplast genomes in the form of fasta coding genes (\emph{i.e.} \textit{exons}).
-\item Extract genes names and apply to solve gene duplication using first method. 
-\item Convert fasta file format to geneVision file format to generate ICM. 
-\item Calculate ICM matrix to find maximum core \textit{Score}. New core genes for two genomes will generate and a specific \textit{CoreId} will assign to it. This process continue until no elements remain in the matrix.
-\item Evolutionary tree will take place by using all data generated from step 1 and 4. The tree will also display the amount of genes lost from each intersection iteration. A specific excel file will be generated that store all the data in local database.
-\end{enumerate}
-
-There main drawback with this method is genes orthography (e.g two different genes sequences with same gene name). In this case, Gene lost is considered by solving gene duplication based on first method to solve gene duplication.
-
-\subsubsection{Core Genes based on Dogma Annotation}  
-The main goal is to get as much as possible the core genes of maximum coding genes names. According to NCBI annotation problem based on \cite{Bakke2009}, annotation method like dogma can give us more reliable coding genes than NCBI. This is because NCBI annotation can carry some annotation and gene identification errors. The general overview of whole process of extraction illustrated in figure \ref{wholesystem}.
-
-extracting core genes based on genes names and counts summarized in the following steps:\\
-\begin{enumerate}
-\item We apply the genome annotation manually using Dogma annotation tool. 
-\item Analysing genomes to store lists of code genes names (\textit{i.e. exons}). solve gene fragments is done by using first method in solve gene fragments. The output from annotation process with dogma is genomes files in GenVision file format. Sets of genes were stored in the database.
-\item Generate ICM matrix to calculate maximum core genes.
-\item Draw the evolutionary tree by extracted all genes sequences from each core. Then applying multiple alignment process on the sequences to calculate the distance among cores to draw a phylogenetic tree.
-
-\end{enumerate}

 
 

-
-The main drawback from the method of extracting core genes based on gene names and counts is that we can not depending only on genes names because of three causes: first, the genome may have not totally named (This can be found in early versions of NCBI genomes), so we will have some lost sequences. Second, we may have two genes sharing the same name, while their sequences are different. Third, we need to annotate all the genomes.               
-
-\subsection{Extract Core Genes based on Gene Quality Control}
-The main idea from this method is to focus on genes quality to predict maximum core genes. By comparing only genes names from one annotation tool is not enough. The question here, does the predicted gene from NCBI is the same gene predicted by Dogma based on gene name and gene sequence?. If yes, then we can predict new quality genomes based on quality control test with a specific threshold. Predicted Genomes comes from merging two annotation techniques. While if no, we can not depending neither on NCBI nor Dogma because of annotation error. Core genes can by predicted by using one of the 
-
-\subsubsection{Core genes based on NCBI and Dogma Annotation}
-This method summarized in the following steps:\\
-
-\begin{enumerate}
-\item Retrieve the annotation of all genomes from NCBI and Dogma: in this step, we apply the annotation of all chloroplast genomes in the database using NCBI annotation and Dogma annotation tool.
-\item Convert NCBI genomes to GeneVision file format, then apply the second method of gene defragmentation methods for NCBI and dogma genomes.  
-\item Predict quality genomes: the process is to pick a genome annotation from two sources, extracting all common genes based on genes names, then applying Needle-man wunch algorithm to align the two sequences based on a threshold equal to 65\%. If the alignment score pass the threshold, then this gene will removed from the competition and store it in quality genome by saving its name with the largest gene sequence with respect to start and end codons. All quality genomes will store in the form of GenVision file format.    
-\item Extract Core genes: from the above two steps, we will have new genomes with quality genes, ofcourse, we have some genes lost here, because dogma produced tRNA and rRNA genes while NCBI did not generate rRNA genes and vise-versa. Build ICM to extract core genes will be sufficient because we already check genes sequences.
-\item Display tree: An evolution tree then will be display based on the intersections of quality genomes.   
-\end{enumerate}