-The field of Genome annotation pays a lot of attentions where the ability to collect and analysis genomical data can provide strong indicator for the study of life\cite{Eisen2007}. Four of genome annotation centers, (such as, \textit{NCBI\cite{Sayers01012011}, Dogma \cite{RDogma}, cpBase \cite{de2002comparative}, CpGAVAS \cite{liu2012cpgavas}, and CEGMA\cite{parra2007cegma}}), present various types of annotations tools (i.e cost-effective sequencing methods\cite{Bakke2009}) on different annotation levels. Generally, one of three methods of gene finding in annotated genome can be categorized using these centers: \textit{alignment-based, composition based, or combination of both\cite{parra2007cegma}}. The alignment-based method is used when we try to predict a coding gene (i.e. genes that produce proteins) by aligning DNA sequence of gene to the protein of cDNA sequence of homology\cite{parra2007cegma}. This approache also is used in GeneWise\cite{birney2004genewise}. Composition-based mothod (known as \textit{ab initio}) is based on a probabilistic model of gene structure to find genes and/or new genes according to the probability gene value (GeneID\cite{parra2000geneid}). In this section, we will consider a new method of finding core genes from large amount of chloroplast genomes, as a solution of the problem resulting from the method stated in section two. This method is based on extracting gene features. A general overview of the system is illustrated in Figure \ref{Fig1}.\\
-\begin{figure}[H]
+These last years the cost of sequencing genomes has been greatly
+reduced, and thus more and more genomes are sequenced. Therefore
+automatic annotation tools are required to deal with this continuously
+increasing amount of genomical data. Moreover, a reliable and accurate
+genome annotation process is needed in order to provide strong
+indicators for the study of life\cite{Eisen2007}.
+
+Various annotation tools (\emph{i.e.}, cost-effective sequencing
+methods\cite{Bakke2009}) producing genomic annotations at many levels
+of detail have been designed by different annotation centers. Among
+the major annotation centers we can notice NCBI\cite{Sayers01012011},
+Dogma \cite{RDogma}, cpBase \cite{de2002comparative},
+CpGAVAS \cite{liu2012cpgavas}, and
+CEGMA\cite{parra2007cegma}. Usually, previous studies used one out of
+three methods for finding genes in annoted genomes using data from
+these centers: \textit{alignment-based}, \textit{composition based},
+or a combination of both~\cite{parra2007cegma}. The alignment-based
+method is used when trying to predict a coding gene (\emph{i.e.}.
+genes that produce proteins) by aligning a genomic DNA sequence with a
+cDNA sequence coding an homologous protein \cite{parra2007cegma}.
+This approach is also used in GeneWise\cite{birney2004genewise}. The
+alternative method, the composition-based one (also known
+as \textit{ab initio}) is based on a probabilistic model of gene
+structure to find genes according to the gene value probability
+(GeneID \cite{parra2000geneid}). Such annotated genomic data will be
+used to overcome the limitation of the first method described in the
+previous section. In fact, the second method we propose finds core
+genes from large amount of chloroplast genomes through genomic
+features extraction.
+
+Figure~\ref{Fig1} presents an overview of the entire method pipeline.
+More precisely, the second method consists of three
+stages: \textit{Genome annotation}, \textit{Core extraction},
+and \textit{Features Visualization} which highlights the
+relationships. To understand the whole core extraction process, we
+describe briefly each stage below. More details will be given in the
+coming subsections. The method uses as starting point some sequence
+database chosen among the many international databases storing
+nucleotide sequences, like the GenBank at NBCI \cite{Sayers01012011},
+the \textit{EMBL-Bank} \cite{apweiler1985swiss} in Europe
+or \textit{DDBJ} \cite{sugawara2008ddbj} in Japan. Different
+biological tools can analyze and annotate genomes by interacting with
+these databases to align and extract sequences to predict genes. The
+database in our method must be taken from any confident data source
+that stores annotated and/or unannotated chloroplast genomes. We have
+considered the GenBank-NCBI \cite{Sayers01012011} database as sequence
+database: 99~genomes of chloroplasts were retrieved. These genomes
+lie in the eleven type of chloroplast families and Table \ref{Tab2}
+summarizes their distribution in our dataset.
+
+\begin{figure}[h]
\centering
- \includegraphics[width=0.7\textwidth]{generalView}
-\caption{A general overview of the system}\label{Fig1}
+ \includegraphics[width=0.75\textwidth]{generalView}
+\caption{A general overview of the annotation-based approach}\label{Fig1}
\end{figure}
-In Figure 1, we illustrate the general overview of system pipeline: \textit{Database, Genomes annotation, Core extraction,} and \textit{relationships}. We will give a short discussion for each stage of the model in order to understand all core extraction process. This work starts with a gene Bank database; however, many international Banks for nucleotide sequence databases (such as, \textit{GenBank} \citep{Sayers01012011} in USA, \textit{EMBL-Bank} \cite{apweiler1985swiss} in Europe, and \textit{DDBJ} \cite{sugawara2008ddbj} in Japon) where exist to store various genomes and DNA species. Different Biological tools are provided to analyse and annotate genomes by interacting with these databases to align and extract sequences to predict genes. The database in this model must be taken from any confident data source that store annotated and/or unannotated chloroplast genomes. We will consider GenBank-NCBI \citep{Sayers01012011} database to be our nucleotide sequences database. Annotation (as the second stage) is considered to be the first important task for Extract Gene Features. Good annotation tool lead us to extracts good gene feature. In this paper, two annotation techniques from \textit{NCBI, and Dogma} used to extract \textit{one genes features}. Extracting Gene feature (as a third stage) can be anything like (genes names, gene sequences, protein sequence,...etc). Our methodologies in this paper consider gene names, gene counts, and gene sequences for extracting core genes and producing chloroplast evolutionary tree. \\
-
-In last stage, to achieve the goal of gene evolution with what the biological expert needs, we used the form of (tables, phylogenetic trees, graphs,...,etc) to organize and represent genomes relationships. In addition, compare these forms with another annotation tool forms for large population of chloroplast genomes give us biological perspective to the nature of chloroplasts evolution. \\
-A Local database attached with each pipe stage is used to store all the informations of extraction process. The output from each stage in our system will be an input to the second stage and so on.
-
-\subsection{Genomes Samples}
-In this research, we retrieved genomes of Chloroplasts from NCBI. Ninety nine genome of them were considered to work with. These genomes lies in the eleven type of chloroplast families, as shown in Table \ref{Tab1}. The distribution of genomes is illustrated in detail in Table \ref{Tab2}.
-
-\input{population_Table}
-
-\subsection{Genome Annotation Techniques}
-Genome annotation is the second stage in the model pipeline. Many techniques were developed to annotate chloroplast genomes but the problem is that they vary in the number and type of predicting genes (i.e the ability to predict genes and \textit{for example: Transfere RNA (tRNA)} and \textit{Ribosomal RNA (rRNA)} genes). Two annotation techniques from NCBI and Dogma are considered to analyse chloroplast genomes to examine the accuracy of predicted coding genes.
-
-\subsubsection{genome annotation from NCBI}
-The objective from this step is to organize genes, solve gene duplications, and generate sets of genes from each genome. The input to the system is our list of chloroplast genomes, annotated from NCBI. All genomes stored as \textit{.fasta} files include collection of protein coding genes\cite{parra2007cegma,RDogma}(gene that produce proteins) with its coding sequences.
-As a preparation step to achieve the set of core genes, we need to analyse these genomes (using \textit{BioPython} package\cite{chapman2000biopython}
-), to extracting all information needed to find the core genes. The process starts by converting each genome from fasta format to GenVision\cite{geneVision} formats from DNASTAR. The outputs from this operation are lists of genes for each genome, their genes names and gene counts. In this stage, we accumulate some Gene duplications for each treated genome. In other words, duplication in gene name can comes from genes fragments as long as chloroplast DNA sequences. We defines \textit{Identical state} to be the state that each gene present only one time in a genome (i.e Gene has no copy) without considering the position or gene orientation. This state can be reached by filtering the database from redundant gene name.
+Annotation, which is the first stage, is an important task for
+extracting gene features. Indeed, to extract good gene feature, a good
+annotation tool is obviously required. To obtain relevant annotated
+genomes, two annotation techniques from NCBI and Dogma are used. The
+extraction of gene feature, the next stage, can be anything like gene
+names, gene sequences, protein sequences, and so on. Our method
+considers gene names, gene counts, and gene sequence for extracting
+core genes and producing chloroplast evolutionary tree. The final
+stage allows to visualize genomes and/or gene evolution in
+chloroplast. Therefore we use representations like tables,
+phylogenetic trees, graphs, etc. to organize and show genomes
+relationships, and thus achieve the goal of representing gene
+evolution. In addition, comparing these representations with ones
+issued from another annotation tool dedicated to large population of
+chloroplast genomes give us biological perspectives to the nature of
+chloroplasts evolution. Notice that a local database linked with each
+pipe stage is used to store all the informations produced during the
+process.
+
+\input{population_Table}
+
+\subsection{Genome annotation techniques}
+
+For the first stage, genome annotation, many techniques have been
+developed to annotate chloroplast genomes. These techniques differ
+from each others in the number and type of predicted genes (for
+example: \textit{Transfer RNA (tRNA)} and \textit{Ribosomal RNA
+(rRNA)} genes). Two annotation techniques from NCBI and Dogma are
+considered to analyze chloroplast genomes.
+
+\subsubsection{Genome annotation from NCBI}
+
+The objective is to generate sets of genes from each genome so that
+genes are organized without any duplication. The input is a list of
+chloroplast genomes annotated from NCBI. More precisely, all genomes
+are stored as \textit{.fasta} files which consists in a collection of
+protein coding genes\cite{parra2007cegma,RDogma} (gene that produce
+proteins) organized in coding sequences. To be able build the set of
+core genes, we need to preprocess these genomes
+using \textit{BioPython} package \cite{chapman2000biopython}. This
+step starts by converting each genome from FASTA file format to
+GenVision \cite{geneVision} format from DNASTAR. Each genome is thus
+converted in a list of genes, with gene names and gene counts. Gene
+name duplications can be accumulated during the treatment of a genome.
+These duplications come from gene fragments (\emph{e.g.} gene
+fragments treated with NCBI) and from chloroplast DNA sequences. To
+ensure that all the duplications are removed, each list of gene is
+translated into a set of genes. Note that NCBI genome annotation
+produces genes except \textit{Ribosomal (rRNA)} genes.
\subsubsection{Genome annotation from Dogma}
-Dogma \cite{RDogma} is an annotation tool developed in the university of Texas in 2004. Dogma is an abbreviation of (\textit{Dual Organellar GenoMe Annotator}) for plant chloroplast and animal mitochondrial genomes.
-It has its own database for translating the genome in all six reading frames and query the amino acid sequence database using Blast\cite{altschul1990basic}(i.e Blastx) with various parameters. Further more, identify protein coding genes\cite{parra2007cegma,RDogma} in the input genome based on sequence similarity of genes in Dogma database. In addition, it can produce the \textit{Transfer RNAs (tRNA)}, and the \textit{Ribosomal RNAs (rRNA)} and verifies their start and end positions rather than NCBI annotation tool. There are no gene duplication with dogma after solving gene fragmentation. \\
-Genome Anntation with dogma can be the key difference of extracting core genes. In figure \ref{dog:Annotation}, The step of annotation divided into two tasks: First, It starts to annotate complete choloroplast genomes (i.e \textit{Unannotated genomes} from NCBI by using Dogma web tool. The whole annotation process was done manually. The output from dogma is considered to be collection of coding genes file for each genome in the form of GeneVision\cite{geneVision} file format.\\
-Where the second task is to solve gene fragments. Defragment process starts immediately after the first task to solve fragments of coding genes for each genome to avoid gene duplication. This process looks for fragment orientation: if it is negative, then the process applis reverse complement operations on gene sequence. All genomes after this stage are fully annotated, their genes were de-fragmented, genes lists and counts were identified.\\
-
-\begin{figure}[H]
- \centering
- \includegraphics[width=0.7\textwidth]{Dogma_GeneName}
- \caption{Dogma Annotation for Chloroplast genomes}\label{dog:Annotation}
-\end{figure}
-
-\subsection{Core Genes Extraction}
-The goal of this step is to extract maximum core genes from sets of genes. The methodology of finding core genes is divided into three methods: \\
-
-The first method is based on extracting core genes by finding common genes feature (i.e Gene names, genes counts). Genomes vary in genes counts according to the annotation used method, so that extracting core genes can be done by constructing Intersection Core Matrix (\textit{ICM}).\\
-While the second method is based on comparing the sequence of reference genes of one annotated genome with other unannotated genomes sequences in Blast database, by using Blastn\cite{Sayers01012011} (nucleotide sequence alignment tool from NCBI). The last method, is based on merge all genes from NCBI and Dogma annotation, then apply a sequence similarity base method (Quality Control test) using Needle-man Wunch algorithm to predict a new genomes. Using predicted genomes to extract core genes using previous methods. Figure \ref{wholesystem}, illustrate the whole system operations.
-
-\begin{figure}[H]
- \centering
- \includegraphics[width=0.7\textwidth]{Whole_system}
- \caption{Total overview of the system pipeline}\label{wholesystem}
-\end{figure}
-
-In the first method, the idea is to iterativelly collect the maximum number of common genes. To do so, the system builds an \textit{Intersection core matrix (ICM)}. ICM is a two dimensional symmetric matrix where each row and column represent one genome. Each position in ICM stores the \textit{intersection scores (IS)}. The Intersection Score is the cardinality number of a core genes comes from intersecting one ????? with other ??????. Maximum cardinality results to select two genomes with their maximum core. Mathematically speaking, if we have an $n \times n$ matrix where $n=\text{number of genomes in local database}$, then lets consider:\\
-
-\begin{equation}
-Score=\max_{i<j}\vert x_i \cap x_j\vert
-\label{Eq1}
-\end{equation}\\
-
-Where $x_i, x_j$ are elements in the matrix. The generation of a new core genes is depending on the cardinality value of intersection elements, we call it $Score$:\\
-$$\text{New Core} = \begin{cases}
-\text{Ignored} & \text{if $Score=0$;} \\
-\text{new Core id} & \text{if $Score>0$.}
-\end{cases}$$\\
-
-if $Score=0$ then we have \textit{disjoint relation} (i.e no common genes between two genomes). In this case the system ignore the vector that smash the core genes. Otherwise, The system will remove these two vectors from ICM and add new core vector with a \textit{coreID} of them to ICM for the calculation in next iteration. The partial core vectors generated with its values will store in the local database for reused to draw the tree. This process repeat until all vectors treated.
-We observe that ICM will result to be very large because of the huge amount of data that it stores. In addition, this will results to be time and memory consuming for calculating the intersection scores by using just genes names. To increase the speed of calculations, we can calculate the upper triangle scores only and exclude diagonal scores. This will reduce whole processing time and memory to half. The time complexity for this process after enhancement changed from $O(n^2-n)$ to $O(\frac{(n-1).n}{2})$. The Algorithm of construction the vector matrix and extracting the vector of maximum core genes where illustrated in Algorithm \ref{Alg1:ICM}. The output from this step is the maximum core vector with its two vectors to draw it in a tree.\\
-
-\begin{algorithm}[H]
-\caption{Extract Maximum Intersection Score}
-\label{Alg1:ICM}
-\begin{algorithmic}
-\REQUIRE $L \leftarrow \text{genomes vectors}$
-\ENSURE $B1 \leftarrow Max core vector$
-\FOR{$i \leftarrow 0:len(L)-1$}
- \STATE $core1 \leftarrow set(GenomeList[L[i]])$
- \STATE $score1 \leftarrow 0$
- \STATE $g1,g2 \leftarrow$ " "
- \FOR{$j \leftarrow i+1:len(L)$}
- \STATE $core2 \leftarrow set(GenomeList[L[i]])$
- \IF{$i < j$}
- \STATE $Core \leftarrow core1 \cap core2$
- \IF{$len(Core) > score1$}
- \STATE $g1 \leftarrow L[i]$
- \STATE $g2 \leftarrow L[j]$
- \STATE $Score \leftarrow len(Core)$
- \ELSIF{$len(Core) == 0$}
- \STATE $g1 \leftarrow L[i]$
- \STATE $g2 \leftarrow L[j]$
- \STATE $Score \leftarrow -1$
- \ENDIF
- \ENDIF
- \ENDFOR
- \STATE $B1[score1] \leftarrow (g1,g2)$
-\ENDFOR
-\RETURN $max(B1)$
-\end{algorithmic}
-\end{algorithm}
-\textit{GenomeList} represents the local database.\\
-In second Method, due to the number of annotated genomes, annotate each genome can be very exhausted task specially with Dogma, because dogma offer a web tool for annotation, so that, each genome must annotate using this web tool. This operation need to do manually. We prefer to recover this problem by choosing one reference chloroplast and querying each reference gene by using \textit{Blastn} to examin its existance in remaining unannotated genomes in blast database. Collect all match genomes from each gene hits, to satisfy the hypothesis "the gene who exists in maximum number of genomes also exist in a core genes". In addition, we can also extract the maximum core genes by examine how many genes present with each genome?. Algorithm \ref{Alg2:secondM}, state the general algorithm for second method. \\
+Dogma stands for \textit{Dual Organellar GenoMe Annotator}. It is an
+annotation tool developed at University of Texas in 2004 for plant
+chloroplast and animal mitochondrial genomes. This tool has its own
+database for translating a genome in all six reading frames and
+queries the amino acid sequence database using
+BLAST \cite{altschul1990basic} (\emph{i.e.} Blastx) with various
+parameters. Protein coding genes are identified in an input genome
+using sequence similarity of genes in Dogma database. In addition in
+comparison with NCBI annotation tool, Dogma can produce
+both \textit{Transfer RNAs (tRNA)} and \textit{Ribosomal RNAs (rRNA)},
+verify their start and end positions. Another difference is also that
+there is no gene duplication with Dogma after solving gene
+fragmentation. In fact, genome annotation with Dogma can be the key
+difference when extracting core genes.
+
+The Dogma annotation process is divided into two tasks. First, we
+manually annotate chloroplast genomes using Dogma web tool. The output
+of this step is supposed to be a collection of coding genes files for
+each genome, organized in GeneVision file. The second task is to solve
+the gene duplication problem and therefore we have use two
+methods. The first method, based on gene name, translates each genome
+into a set of genes without duplicates. The second method avoid gene
+duplication through a defragment process. In each iteration, this
+process starts by taking a gene from gene list, searches for gene
+duplication, if a duplication is found, it looks on the orientation of
+the fragment sequence. If it is positive it appends directly the
+sequence to gene files. Otherwise reverse complement operations are
+applied on the sequence, which is then also append to gene files.
+Finally, a check for missing start and stop codons is performed. At
+the end of the annotation process, all the genomes are fully
+annotated, their genes are defragmented, and gene counts are
+available.
+
+\subsection{Core genes extraction}
+
+The goal of this stage is to extract maximum core genes from sets of
+genes. To find core genes, the following methodology is applied.
+
+\subsubsection{Preprocessing}
+
+In order to extract core genomes in a suitable manner, the genomic
+data are preprocessed with two methods: on the one hand a method based
+on gene name and count, and on the other hand a method based on a
+sequence quality control test.
+
+In the first method, we extract a list of genes from each chloroplast
+genome. Then we store this list of genes in the database under genome
+nam and genes counts can be extracted by a specific length command.
+The \textit{Intersection Core Matrix}, described in next subsection,
+is then computed to extract the core genes. The problem with this
+method can be stated as follows: how can we ensure that the gene which
+is predicted in core genes is the same gene in leaf genomes? The
+answer to this problem is that if the sequences of any gene in a
+genome annotated from Dogma and NCBI are similar with respect to a
+given threshold, then we do not have any problem with this
+method. When the sequences are not similar we have a problem, because
+we cannot decide which sequence belongs to a gene in core genes.
+
+The second method is based on the underlying idea: we can predict the
+the best annotated genome by merging the annotated genomes from NCBI
+and Dogma according to a quality test on genes names and sequences. To
+obtain all quality genes of each genome, we consider the following
+hypothesis: any gene will appear in the predicted genome if and only
+if the annotated genes in NCBI and Dogma pass a specific threshold
+of \textit{quality control test}. In fact, the Needle-man Wunch
+algorithm is applied to compare both sequences with respect to a
+threshold. If the alignment score is above the threshold, then the
+gene will be retained in the predicted genome, otherwise the gene is
+ignored. Once the prediction of all genomes is done,
+the \textit{Intersection Core Matrix} is computed on these new genomes
+to extract core genes, as explained in Algorithm \ref{Alg3:thirdM}.
\begin{algorithm}[H]
-\caption{Extract Maximum Core genes based on Blast}
-\label{Alg2:secondM}
-\begin{algorithmic}
-\REQUIRE $Ref\_Genome \leftarrow \text{Accession No}$
-\ENSURE $core \leftarrow \text{Genomes for each gene}$
-\FOR{$gene \leftarrow Ref\_Genome$}
- \STATE $G\_list= \text{empty list}$
- \STATE $File \leftarrow Blastn(gene)$
- \STATE $G\_list \leftarrow File[\text{Genomes names}]$
- \STATE $Core \leftarrow [Accession\_No:G\_list]$
-\ENDFOR
-\RETURN $Core$
-\end{algorithmic}
-\end{algorithm}
-
-The hypothesis in last method state: we can predict the best annotated genome by merge the annotated genomes from NCBI and dogma based on the quality of genes names and sequences. To generate all quality genes of each genome. the hypothesis state: Any gene will be in predicted genome if and only if the annotated genes between NCBI and Dogma pass a specific threshold of\textit{quality control test}. To accept the quality test, we applied Needle-man Wunch algorithm to compare two gene sequences with respect to pass a threshold. If the alignment score pass this threshold, then the gene will be in the predicted genome, else the gene will be ignored. After predicting all genomes, one of previous two methods can be applied to extract core genes. As shown in Algorithm \ref{Alg3:thirdM}.
-
-\begin{algorithm}[H]
-\caption{Extract new genome based on Gene Quality test}
+\caption{Extract new genome based on gene quality test}
\label{Alg3:thirdM}
\begin{algorithmic}
\REQUIRE $Gname \leftarrow \text{Genome Name}, Threshold \leftarrow 65$
\end{algorithmic}
\end{algorithm}
-Here, geneChk is a subroutine in python, it is used to find the best similarity score between two gene sequences after applying operations like \textit{reverse, complement, and reverse complement}. The algorithm of geneChk is illustrated in Algorithm \ref{Alg3:genechk}.
+\textbf{geneChk} is a subroutine used to find the best similarity score between
+two gene sequences after applying operations like \textit{reverse}, {\it complement},
+and {\it reverse complement}. Algorithm~\ref{Alg3:genechk} gives the outline of
+geneChk subroutine.
\begin{algorithm}[H]
-\caption{Find the Maximum similarity score between two sequences}
+\caption{Find the Maximum Similarity Score between two sequences}
\label{Alg3:genechk}
\begin{algorithmic}
-\REQUIRE $gen1,gen2 \leftarrow \text{NCBI gene sequence, Dogma gene sequence}$
+\REQUIRE $g1,g2 \leftarrow \text{NCBI gene sequence, Dogma gene sequence}$
\ENSURE $\text{Maximum similarity score}$
-\STATE $Score1 \leftarrow needle(gen1,gen2)$
-\STATE $Score2 \leftarrow needle(gen1,Reverse(gen2))$
-\STATE $Score3 \leftarrow needle(gen1,Complement(gen2))$
-\STATE $Score4 \leftarrow needle(gen1,Reverse(Complement(gen2)))$
-\IF {$max(Score1, Score2, Score3, Score4)==Score1$}
- \RETURN $Score1$
-\ELSIF {$max(Score1, Score2, Score3, Score4)==Score2$}
- \RETURN $Score2$
-\ELSIF {$max(Score1, Score2, Score3, Score4)==Score3$}
- \RETURN $Score3$
-\ELSIF {$max(Score1, Score2, Score3, Score4)==Score4$}
- \RETURN $Score4$
-\ENDIF
+\STATE $score1 \leftarrow needle(g1,g2)$
+\STATE $score2 \leftarrow needle(g1,Reverse(g2))$
+\STATE $score3 \leftarrow needle(g1,Complement(g2))$
+\STATE $score4 \leftarrow needle(g1,Reverse(Complement(g2)))$
+\RETURN $max(score1,score2,score3,score4)$
\end{algorithmic}
\end{algorithm}
-\subsection{Visualizing Relationships}
-The goal here is to visualizing the results by build a tree of evolution. The system can produce this tree automatically by using Dot graphs package\cite{gansner2002drawing} from Graphviz library and all information available in a database. Core genes generated with their genes can be very important information in the tree, because they can viewed as an ancestor information for two genomes or more. Further more, each node represents a genome or core as \textit{(Genes count:Family name, Scientific names, Accession number)}, Edges represent numbers of lost genes from genomes-core or core-core relationship. The number of lost genes here can represent an important factor for evolution, it represents how much lost of genes for the species in same or different families. By the principle of classification, small number of gene lost among species indicate that those species are close to each other and belong to same family, while big genes lost means that species is far to be in the same family. To see the picture clearly, Phylogenetic tree is an evolutionary tree generated also by the system. Generating this tree is based on the distances among genes sequences. There are many resources to build such tree (for example: PHYML\cite{guindon2005phyml}, RAxML{\cite{stamatakis2008raxml,stamatakis2005raxml}, BioNJ , and TNT\cite{goloboff2008tnt}}. We consider to use RAxML\cite{stamatakis2008raxml,stamatakis2005raxml} to generate this tree.
-
-\section{Implementation}
-We implemented four algorithms to extract maximum core genes from large amount of chloroplast genomes. Two algorithms used to extract core genes based on NCBI annotation, and the others based on dogma annotation tool. Evolutionary tree generated as a result from each method implementation. In this section, we will present the four methods, and how they can extract maximum core genes?, and how the developed code will generate the evolutionary tree.
-
-\subsection{Extract Core Genes based on Gene Contents}
-
-\subsubsection{Core Genes based on NCBI Annotation}
-The first idea to construct the core genome is based on the extraction of Genes names (as gene presence or absence). For instant, in this stage neither sequence comparison nor new annotation were made, we just want to extract all genes with counts stored in each chloroplast genome, then find the intersection core genes based on gene names. \\
-The pipeline of extracting core genes can summarize in the following steps:\\
-First, we apply the genome annotation method using NCBI annotation tool. Genome quality check can be used in this step to ensure that genomes pass some quality condition. Then, the system lunch annotation process using NCBI to extract code genes (i.e \textit{exons}) and solve gene fragments. From NCBI, we did not observe any problem with genes fragments, but there are a problem of genes orthography (e.g two different genes sequences with same gene name). After we obtain all annotated genomes from NCBI to the local database, the code will then automatically will generate GenVision\cite{geneVision} file format to lunch the second step to extract coding genes names and counts. The competition will start by building intersection matrix to intersect genomes vectors in the local database with the others. New core vector for two leaf vectors will generate and a specific \textit{CoreId} will assign to it. an evolutionary tree will take place by using all data generated from step 1 and 2. The tree will also display the amount of genes lost from each intersection iteration. A specific excel file will be generated that store all the data in local database. The whole operation illstrate in Figure \ref{NCBI:geneextraction}.
+\subsubsection{Intersection Core Matrix (\textit{ICM})}
+
+To extract core genes, we iteratively collect the maximum number of
+common genes between genomes and therefore during this stage
+an \textit{Intersection Core Matrix} (ICM) is built. ICM is a two
+dimensional symmetric matrix where each row and each column correspond
+to one genome. Hence, an element of the matrix stores
+the \textit{Intersection Score} (IS): the cardinality of the core
+genes set obtained by intersecting one genome with another
+one. Maximum cardinality results in selecting the two genomes having
+the maximum score. Mathematically speaking, if we have $n$ genomes in
+local database, the ICM is an $n \times n$ matrix whose elements
+satisfy:
+\begin{equation}
+score_{ij}=\vert g_i \cap g_j\vert
+\label{Eq1}
+\end{equation}
+\noindent where $1 \leq i \leq n$, $1 \leq j \leq n$, and $g_i, g_j$ are
+genomes. The generation of a new core gene depends obviously on the
+value of the intersection scores $score_{ij}$. More precisely, the
+idea is to consider a pair of genomes such that their score is the
+largest element in ICM. These two genomes are then removed from matrix
+and the resulting new core genome is added for the next iteration.
+The ICM is then updated to take into account the new core gene: new IS
+values are computed for it. This process is repeated until no new core
+gene can be obtained.
+
+We can observe that the ICM is very large due to the amount of
+data. As a consequence, the computation of the intersection scores is
+both time and memory consuming. However, since ICM is a symetric
+matrix we can reduce the computation overhead by considering only its
+triangular upper part. The time complexity for this process after
+enhancement is thus $O(\frac{n.(n-1)}{2})$. Algorithm ~\ref{Alg1:ICM}
+illustrates the construction of the ICM matrix and the extraction of
+the core genes, where \textit{GenomeList} represents the database
+storing all genomes data. At each iteration, it computes the maximum
+core genes with its two genomes parents.
+
+% ALGORITHM HAS BEEN REWRITTEN
-\begin{figure}[H]
- \centering
- \includegraphics[width=0.7\textwidth]{NCBI_geneextraction}
- \caption{Extract core genes based on NCBI gene names and counts}\label{NCBI:geneextraction}
-\end{figure}
+\begin{algorithm}[H]
+\caption{Extract Maximum Intersection Score}
+\label{Alg1:ICM}
+\begin{algorithmic}
+\REQUIRE $L \leftarrow \text{genomes sets}$
+\ENSURE $B1 \leftarrow \text{Max Core set}$
+\FOR{$i \leftarrow 1:len(L)-1$}
+ \STATE $score \leftarrow 0$
+ \STATE $core1 \leftarrow set(GenomeList[L[i]])$
+ \STATE $g1 \leftarrow L[i]$
+ \FOR{$j \leftarrow i+1:len(L)$}
+ \STATE $core2 \leftarrow set(GenomeList[L[j]])$
+ \STATE $core \leftarrow core1 \cap core2$
+ \IF{$len(core) > score$}
+ \STATE $score \leftarrow len(core)$
+ \STATE $g2 \leftarrow L[j]$
+ \ENDIF
+ \ENDFOR
+ \STATE $B1[score] \leftarrow (g1,g2)$
+\ENDFOR
+\RETURN $max(B1)$
+\end{algorithmic}
+\end{algorithm}
-\subsubsection{Core Genes based on Dogma Annotation}
-The main goal is to get as much as possible the core genes of maximum coding genes names. According to NCBI annotation problem based on \cite{Bakke2009}, annotation method like dogma can give us more reliable coding genes than NCBI. This is because NCBI annotation can carry some annotation and gene identification errors. The general overview of whole process of extraction illustrated in figure \ref{dog:Annotation}. From this figure, the pipeline of extracting core genes can summarize in the following steps:\\
-First, we apply the genome annotation method using Dogma annotation tool. Genome quality check can be used in this step to ensure that genomes pass some quality condition. Then, the system lunch annotation process using Dogma to extract code genes (i.e \textit{exons}) and solve gene fragments. The key difference here is that dogma can generate in addition transfer RNA and ribosomal RNA. As a result from annotation process with dogma is genomes files in GenVision\cite{geneVision} file format, the code will lunch genes de-fragments process to avoid genes duplications. little problems of genes orthography (e.g two different genes sequences with same gene name) where exists. After we obtain all annotated genomes from dogma, we store it in the local database. The code will then automatically lunch the second step to extract coding genes names and counts. The competition will start by building intersection matrix to intersect genomes vectors in the local database with the others. New core vector for two leaf vectors will generate and a specific \textit{CoreId} will assign to it. an evolutionary tree will take place by using all data generated from step 1 and 2. The tree will also display the amount of genes lost from each intersection iteration. A specific excel file will be generated that store all the data in local database. The whole operation illustrate in Figure \ref{dogma:geneextraction}.
+\subsection{Features visualization}
+
+The goal is to visualize results by building a tree of evolution. All
+core genes generated represent an important information in the tree,
+because they provide ancestor information of two or more
+genomes. Each node in the tree represents one chloroplast genome or
+one predicted core and labelled as \textit{(Genes count:Family name\_Scientific
+names\_Accession number)}. While an edge is labelled with the number of
+lost genes from a leaf genome or an intermediate core gene. Such
+numbers are very interesting because they give an information about
+the evolution: how many genes were lost between two species whether
+they belong to the same family or not. By the principle of
+classification, a small number of genes lost among species indicates
+that those species are close to each other and belong to same family,
+while a large lost means that we have an evolutionary relationship
+between species from different families. To depict the links between
+species clearly, we built a phylogenetic tree showing the
+relationships based on the distances among genes sequences. Many tools
+are available to obtain a such tree, for example:
+PHYML\cite{guindon2005phyml},
+RAxML{\cite{stamatakis2008raxml,stamatakis2005raxml}, BioNJ, and
+TNT\cite{goloboff2008tnt}}. In this work, we chose to use
+RAxML\cite{stamatakis2008raxml,stamatakis2005raxml} because it is
+fast, accurate, and can build large trees when dealing with a large
+number of genomic sequences.
+
+The procedure used to built a phylogenetic tree is as follows:
+\begin{enumerate}
+\item For each gene in a core gene, extract its sequence and store it in the database.
+\item Use multiple alignment tools such as (****to be write after see christophe****)
+to align these sequences with each others.
+\item Use an outer-group genome from cyanobacteria to calculate distances.
+\item Submit the resulting aligned sequences to RAxML program to compute
+the distances and finally draw the phylogenetic tree.
+\end{enumerate}
\begin{figure}[H]
- \centering
- \includegraphics[width=0.7\textwidth]{Dogma_geneextraction}
- \caption{Extract core genes based on Dogma gene names and counts}\label{dogma:geneextraction}
+ \centering \includegraphics[width=0.75\textwidth]{Whole_system} \caption{Overview
+ of the pipeline}\label{wholesystem}
\end{figure}
-The main drawback from the method of extracting core genes based on gene names and counts is that we can not depending only on genes names because of three causes: first, the genome may have not totally named (This can be found in early versions of NCBI genomes), so we will have some lost sequences. Second, we may have two genes sharing the same name, while their sequences are different. Third, we need to annotate all the genomes.
-
-\subsection{Extract Core Genes based on Genes Sequences}
-We discussed before on the hypothesis of the second method. In this section, we will implement this hypothesis by using ncbi-Blast alignment tool. Implementation of this method is dividing into two parts: \textit{Core genes from NCBI Annotation} and \textit{Core Genes from Dogma Annotation}. For instance, for the two parts, selecting a reference genome can be a key difference among predicting Core genes. After choosing a reference genome, Local blast database will then created to store the rest of Un-annotated chloroplast genomes. \\
+\section{Implementation}
-We will present the algorithm in the following steps:
+The different algorithms have been implemented using Python version
+2.7, on a laptop running Ubuntu~12.04~LTS. More precisely, the
+computer is a Dell Latitude laptop - model E6430 with 6~GiB memory and
+a quad-core Intel core~i5~processor with an operating frequency of
+2.5~GHz. Many python packages such as os, Biopython, memory\_profile,
+re, numpy, time, shutil, and xlsxwriter were used to extract core
+genes from large amount of chloroplast genomes.
+
+\begin{center}
+\begin{table}[b]
+\caption{Type of annotation, execution time, and core genes
+for each method}\label{Etime}
+{\scriptsize
+\begin{tabular}{p{2cm}p{0.5cm}p{0.25cm}p{0.5cm}p{0.25cm}p{0.5cm}p{0.25cm}p{0.5cm}p{0.25cm}p{0.5cm}p{0.2cm}}
+\hline\hline
+ Method & \multicolumn{2}{c}{Annotation} & \multicolumn{2}{c}{Features} & \multicolumn{2}{c}{Exec. time (min.)} & \multicolumn{2}{c}{Core genes} & \multicolumn{2}{c}{Bad genomes} \\
+~ & N & D & Name & Seq & N & D & N & D & N & D \\
+\hline
+Gene prediction & $\surd$ & - & - & $\surd$ & ? & - & ? & - & 0 & -\\[0.5ex]
+Gene features & $\surd$ & $\surd$ & $\surd$ & - & 4.98 & 1.52 & 28 & 10 & 1 & 0\\[0.5ex]
+Gene quality & $\surd$ & $\surd$ & $\surd$ & $\surd$ & \multicolumn{2}{c}{$\simeq$3 days + 1.29} & \multicolumn{2}{c}{4} & \multicolumn{2}{c}{1}\\[1ex]
+\hline
+\end{tabular}
+}
+\end{table}
+\end{center}
+
+\vspace{-1cm}
+
+Table~\ref{Etime} presents for each method the annotation type,
+execution time, and the number of core genes. We use the following
+notations: \textbf{N} denotes NCBI, while \textbf{D} means DOGMA,
+and \textbf{Seq} is for sequence. The first {\it Annotation} columns
+represent the algorithm used to annotate chloroplast genomes, the {\it
+Features} columns mean the kind of gene feature used to extract core
+genes: gene name, gene sequence, or both of them. It can be seen that
+almost all methods need low {\it Execution time} to extract core genes
+from large chloroplast genome. Only the gene quality method requires
+several days of computation (about 3-4 days) for sequence comparisons,
+once the quality genomes are construced it takes just 1.29~minutes to
+extract core gene. Thanks to this low execution times we can use these
+methods to extract core genes on a personal computer rather than main
+frames or parallel computers. The lowest execution time: 1.52~minutes,
+is obtained with the second method using Dogma annotations. The number
+of {\it Core genes} represents the amount of genes in the last core
+genome. The main goal is to find the maximum core genes that simulate
+biological background of chloroplasts. With NCBI we have 28 genes for
+96 genomes, instead of 10 genes for 97 genomes with
+Dogma. Unfortunately, the biological distribution of genomes with NCBI
+in core tree do not reflect good biological perspective, whereas with
+DOGMA the distribution of genomes is biologically relevant. {\it Bad
+genomes} gives the number of genomes that destroy core genes due to
+low number of gene intersection. \textit{NC\_012568.1 Micromonas
+pusilla} is the only genome which destroyed the core genome with NCBI
+annotations for both gene features and gene quality methods.
+
+The second important factor is the amount of memory being used by each
+methodology. Table \ref{mem} shows the memory usage of each
+method. We used a package from PyPI~(\textit{the Python Package
+Index}) named \textit{Memory\_profile} (located at~{\tt
+https://pypi.python.org/pypi}) to extract all the values in
+table~\ref{mem}. In this table, the values are presented in megabyte
+unit and \textit{gV} means genevision~file~format. We can notice that
+the level of memory which is used is relatively low for all methods
+and is available on any personal computer. The different values also
+show that the gene features method based on Dogma annotations has the
+more reasonable memory usage, except when extracting core
+sequences. The third method gives the lowest values if we already have
+the quality genomes, otherwise it will consume far more
+memory. Moreover, the amount of memory used by the third method also
+depends on the size of each genome.
+
+\begin{center}
+\begin{table}[H]
+\caption{Memory usages in (MB) for each methodology}\label{mem}
+{\scriptsize
+\begin{tabular}{p{2.5cm}p{1.5cm}p{1cm}p{1cm}p{1cm}p{1cm}p{1cm}p{1cm}}
+\hline\hline
+Method& & Load Gen. & Conv. gV & Read gV & ICM & Core tree & Core Seq. \\
+\hline
+Gene prediction & ~ & ~ & ~ & ~ & ~ & ~ & ~\\
+\multirow{2}{*}{Gene Features} & NCBI & 15.4 & 18.9 & 17.5 & 18 & 18 & 28.1\\
+ & DOGMA& 15.3 & 15.3 & 16.8 & 17.8 & 17.9 & 31.2\\
+Gene Quality & ~ & 15.3 & $\le$3G & 16.1 & 17 & 17.1 & 24.4\\
+\hline
+\end{tabular}
+}
+\end{table}
+\end{center}
-\begin{enumerate}
-\item Select a reference genome: we need to select good reference genome from our population, To do so, we can choose \textit{Lycopersicon esculentum cultivar LA3023 chloroplast NC\_007898.3} to be the reference genome if we consider the version of annotation, or \textit{Zea Mays NC\_001666.2} if we consider the largest number of coding genes based on NCBI annotation.The aim is to extract the maximum core genes. In order to achieve this goal, we choose \textit{Zea Mays NC\_001666.2} to be our reference genome.
-\item Build Blast database for the rest of unannotated genomes.
-\item Compare reference Genes: based on the genomes in the database. We querying each reference gene with the database by using \textbf{Blastn}. The result with alignment scores for each gene will store in separated file.
-\item Generate match table: In this table, each row represent referenced genes, while columns represent genomes. To fill this table, a developed code will open each output file for reference genes and extract the number of genomes and a list of genomes names where gene sequence have hits.
-\end{enumerate}
-The core genome can be extracted from the table by taking as possible the maximum number of genes that exists in the maximum number of genomes.
-\subsection{Extract Core Genes based on Gene Quality Control}
-The main idea from this method is to focus on genes quality to predict maximum core genes. By comparing only genes names or genes sequences from one annotation tool is not enough. The question here, does the predicted gene from NCBI is the same gene predicted by Dogma based on gene name and gene sequence?. If yes, then we can predict new quiality genomes based on quality control test with a specific threshold. Predicted Genomes comes from merging two annotation techniques. While if no, we can not depending neither on NCBI nor Dogma because of annotation error. Core genes can by predicted by using one of the previous methods.
-This method summarized in the following steps:\\
-\begin{enumerate}
-\item Retrieve the annotation of all genomes from NCBI and Dogma: in this step, we apply the annotation of all chloroplast genomes in the database using NCBI annotation and Dogma annotation tool.
-\item Predict quality genomes: the process is to pick a genome annotation from two techniques, extracting all common genes based on genes names, then applying Needle-man wunch algorithm to align the two sequences based on a specific threshold. If the alignment score pass the threshold, then this gene will removed from the competition and store it in quality genome by saving its name with the largest gene sequence with respect to start and end codons. All quality genomes will store in the form of GenVision file format.
-\item Extract Core genes: from the above two steps, we will have new genomes with quality genes, ofcourse, we have some genes lost here, because dogma produced tRNA and rRNA genes while NCBI did not generate them and vise-versa. Using first method to extract core genes will be sufficient because we already check their sequences.
-\item Display tree: An evolution tree then will be display based on the intersections of quality genomes.
-\end{enumerate}
-\pagebreak
\ No newline at end of file
