\input{population_Table}
-% MICHEL : TO BE CONTINUED FROM HERE
+\subsection{Genome annotation techniques}
-\subsection{Genome Annotation Techniques}
-Genome annotation is the second stage in the model pipeline. Many techniques were developed to annotate chloroplast genomes but the problem is that they vary in the number and type of predicted genes (\emph{i.e.} the ability to predict genes and \textit{for example: Transfer RNA (tRNA)} and \textit{Ribosomal RNA (rRNA)} genes). Two annotation techniques from NCBI and Dogma are considered to analyse chloroplast genomes to examine the accuracy of predicted coding genes.
+For the first stage, genome annotation, many techniques have been
+developed to annotate chloroplast genomes. These techniques differ
+from each others in the number and type of predicted genes (for
+example: \textit{Transfer RNA (tRNA)} and \textit{Ribosomal RNA
+(rRNA)} genes). Two annotation techniques from NCBI and Dogma are
+considered to analyze chloroplast genomes.
\subsubsection{Genome annotation from NCBI}
-The objective from this step is to organize genes, solve gene duplications, and generate sets of genes from each genome. The input to the system is a list of chloroplast genomes, annotated from NCBI. All genomes stored as \textit{.fasta} files which have a collection of protein coding genes\cite{parra2007cegma,RDogma} (gene that produce proteins) with its coding sequences.
-As a preprocessing step to build the set of core genes, we need to analyse these genomes (using \textit{BioPython} package\cite{chapman2000biopython}). The process starts by converting each genome from fasta format to GenVision\cite{geneVision} format from DNASTAR. The outputs from this operation are lists of genes for each genome, their gene names and gene counts. In this stage, we accumulate some gene duplications for each treated genome. These gene name duplication can come from gene fragments, (e.g. gene fragments treated with NCBI), and from chloroplast DNA sequences. To ensure that all the duplications are removed, each list of genes is translated into a set of genes. NCBI genome annotation produce genes except \textit{Ribosomal rRNA}.
-\subsubsection{Genome annotation from Dogma}
-Dogma is an annotation tool developed in the university of Texas in 2004. Dogma is an abbreviation of (\textit{Dual Organellar GenoMe Annotator}) for plant chloroplast and animal mitochondrial genomes.
-It has its own database for translating the genome in all six reading frames and it queries the amino acid sequence database using Blast\cite{altschul1990basic}(\emph{i.e.} Blastx) with various parameters. Furthermore, identify protein coding genes in the input genome based on sequence similarity of genes in Dogma database. In addition, it can produce the \textit{Transfer RNAs (tRNA)}, and the \textit{Ribosomal RNAs (rRNA)} and verifies their start and end positions rather than NCBI annotation tool. There is no gene duplication with dogma after solving gene fragmentation. \\
-Genome annotation with dogma can be the key difference of extracting core genes. The step of annotation is divided into two tasks: first, It starts to annotate complete chloroplast genomes (\emph{i.e.} \textit{Unannotate genome from NCBI} by using Dogma web tool. This process is done manually. The output from dogma is considered to be a collection of coding genes files for each genome in the form of GeneVision file format.
-The second task is to solve gene fragments. Two methods are used to solve gene duplication. First, for the method based on gene name, all the duplications are removed, where each list of genes is translated into a set of genes. Second, for the method of gene quality test, a defragment process used to avoid gene duplication. \\
-In each iteration, this process starts by taking one gene from gene list, searches for gene duplication, if exists, it looks on the orientation of the fragment sequence: if it is positive, then it appends fragment sequence to a gene files. Otherwise, the process applies reverse complement operations on gene sequences and appends it to gene files. An additional process is then applied to check start and stop codons in case of missing. All genomes after this stage are fully annotated, their genes are de-fragmented, and counts are identified.\\
+The objective is to generate sets of genes from each genome so that
+genes are organized without any duplication. The input is a list of
+chloroplast genomes annotated from NCBI. More precisely, all genomes
+are stored as \textit{.fasta} files which consists in a collection of
+protein coding genes\cite{parra2007cegma,RDogma} (gene that produce
+proteins) organized in coding sequences. To be able build the set of
+core genes, we need to preprocess these genomes
+using \textit{BioPython} package \cite{chapman2000biopython}. This
+step starts by converting each genome from FASTA file format to
+GenVision \cite{geneVision} format from DNASTAR. Each genome is thus
+converted in a list of genes, with gene names and gene counts. Gene
+name duplications can be accumulated during the treatment of a genome.
+These duplications come from gene fragments (\emph{e.g.} gene
+fragments treated with NCBI) and from chloroplast DNA sequences. To
+ensure that all the duplications are removed, each list of gene is
+translated into a set of genes. Note that NCBI genome annotation
+produces genes except \textit{Ribosomal (rRNA)} genes.
-\subsection{Core Genes Extraction}
-The goal of this step is to extract maximum core genes from sets of genes. The methodology of finding core genes is as follow: \\
+\subsubsection{Genome annotation from Dogma}
-\subsubsection{Pre-Processing}
-We apply two pre-processing methods to organize and prepare genomes data: the first method based on gene name and count, and the second one is based on sequence quality control test.\\
-In the first method, we extract a list of genes from each chloroplast genome. Then we store this list of genes in the database under genome name. Genes counts can be extracted by a specific length command. \textit{Intersection Core Matrix} then applied to extract the core genes. The problem with this method is how can we ensure that the gene which is predicted in core genes is the same gene in leaf genomes? The answer of this question is as follows: if the sequence of any gene in a genome annotated from dogma and NCBI are similar with respect to a threshold, we do not have any problem with this method. Otherwise, we have a problem, because we can not decide which sequence goes to a gene in core genes.
-The second pre-processing method states: we can predict the best annotated genome by merging the annotated genomes from NCBI and dogma if we follow the quality of genes names and sequences test. To generate all quality genes of each genome, the hypothesis state: any gene will be in predicted genome if and only if the annotated genes between NCBI and Dogma pass a specific threshold of \textit{quality control test}. To accept the quality test, we applied Needle-man Wunch algorithm to compare two gene sequences with respect to a threshold. If the alignment score passes the threshold, then the gene will be in the predicted genome. Otherwise, the gene is ignored. After predicting all genomes, \textit{Intersection Core Matrix} is applied on these new genomes to extract core genes, as shown in Algorithm \ref{Alg3:thirdM}.
+Dogma stands for \textit{Dual Organellar GenoMe Annotator}. It is an
+annotation tool developed at University of Texas in 2004 for plant
+chloroplast and animal mitochondrial genomes. This tool has its own
+database for translating a genome in all six reading frames and
+queries the amino acid sequence database using
+BLAST \cite{altschul1990basic} (\emph{i.e.} Blastx) with various
+parameters. Protein coding genes are identified in an input genome
+using sequence similarity of genes in Dogma database. In addition in
+comparison with NCBI annotation tool, Dogma can produce
+both \textit{Transfer RNAs (tRNA)} and \textit{Ribosomal RNAs (rRNA)},
+verify their start and end positions. Another difference is also that
+there is no gene duplication with Dogma after solving gene
+fragmentation. In fact, genome annotation with Dogma can be the key
+difference when extracting core genes.
+
+The Dogma annotation process is divided into two tasks. First, we
+manually annotate chloroplast genomes using Dogma web tool. The output
+of this step is supposed to be a collection of coding genes files for
+each genome, organized in GeneVision file. The second task is to solve
+the gene duplication problem and therefore we have use two
+methods. The first method, based on gene name, translates each genome
+into a set of genes without duplicates. The second method avoid gene
+duplication through a defragment process. In each iteration, this
+process starts by taking a gene from gene list, searches for gene
+duplication, if a duplication is found, it looks on the orientation of
+the fragment sequence. If it is positive it appends directly the
+sequence to gene files. Otherwise reverse complement operations are
+applied on the sequence, which is then also append to gene files.
+Finally, a check for missing start and stop codons is performed. At
+the end of the annotation process, all the genomes are fully
+annotated, their genes are defragmented, and gene counts are
+available.
+
+\subsection{Core genes extraction}
+
+The goal of this stage is to extract maximum core genes from sets of
+genes. To find core genes, the following methodology is applied.
+
+\subsubsection{Preprocessing}
+
+In order to extract core genomes in a suitable manner, the genomic
+data are preprocessed with two methods: on the one hand a method based
+on gene name and count, and on the other hand a method based on a
+sequence quality control test.
+
+In the first method, we extract a list of genes from each chloroplast
+genome. Then we store this list of genes in the database under genome
+nam and genes counts can be extracted by a specific length command.
+The \textit{Intersection Core Matrix}, described in next subsection,
+is then computed to extract the core genes. The problem with this
+method can be stated as follows: how can we ensure that the gene which
+is predicted in core genes is the same gene in leaf genomes? The
+answer to this problem is that if the sequences of any gene in a
+genome annotated from Dogma and NCBI are similar with respect to a
+given threshold, then we do not have any problem with this
+method. When the sequences are not similar we have a problem, because
+we cannot decide which sequence belongs to a gene in core genes.
+
+The second method is based on the underlying idea: we can predict the
+the best annotated genome by merging the annotated genomes from NCBI
+and Dogma according to a quality test on genes names and sequences. To
+obtain all quality genes of each genome, we consider the following
+hypothesis: any gene will appear in the predicted genome if and only
+if the annotated genes in NCBI and Dogma pass a specific threshold
+of \textit{quality control test}. In fact, the Needle-man Wunch
+algorithm is applied to compare both sequences with respect to a
+threshold. If the alignment score is above the threshold, then the
+gene will be retained in the predicted genome, otherwise the gene is
+ignored. Once the prediction of all genomes is done,
+the \textit{Intersection Core Matrix} is computed on these new genomes
+to extract core genes, as explained in Algorithm \ref{Alg3:thirdM}.
\begin{algorithm}[H]
-\caption{Extract new genome based on Gene Quality test}
+\caption{Extract new genome based on gene quality test}
\label{Alg3:thirdM}
\begin{algorithmic}
\REQUIRE $Gname \leftarrow \text{Genome Name}, Threshold \leftarrow 65$
\STATE $geneList=\text{empty list}$
\STATE $common=set(dir(NCBI\_Genes)) \cap set(dir(Dogma\_Genes))$
\FOR{$\text{gene in common}$}
- \STATE $g1 \leftarrow open(NCBI\_Genes(gene)).read()$
- \STATE $g2 \leftarrow open(Dogma\_Genes(gene)).read()$
- \STATE $score \leftarrow geneChk(g1,g2)$
+ \STATE $gen1 \leftarrow open(NCBI\_Genes(gene)).read()$
+ \STATE $gen2 \leftarrow open(Dogma\_Genes(gene)).read()$
+ \STATE $score \leftarrow geneChk(gen1,gen2)$
\IF {$score > Threshold$}
\STATE $geneList \leftarrow gene$
\ENDIF
\end{algorithmic}
\end{algorithm}
-\textbf{geneChk} is a subroutine, it is used to find the best similarity score between two gene sequences after applying operations like \textit{reverse, complement, and reverse complement}. The algorithm of geneChk is illustrated in Algorithm \ref{Alg3:genechk}.
+\textbf{geneChk} is a subroutine used to find the best similarity score between
+two gene sequences after applying operations like \textit{reverse}, {\it complement},
+and {\it reverse complement}. Algorithm~\ref{Alg3:genechk} gives the outline of
+geneChk subroutine.
\begin{algorithm}[H]
-\caption{Find the Maximum similarity score between two sequences}
+\caption{Find the maximum similarity score between two sequences}
\label{Alg3:genechk}
\begin{algorithmic}
\REQUIRE $gen1,gen2 \leftarrow \text{NCBI gene sequence, Dogma gene sequence}$
\subsubsection{Intersection Core Matrix (\textit{ICM})}
-The idea behind extracting core genes is to iteratively collect the maximum number of common genes between two genomes. To do so, the system builds an \textit{Intersection Core Matrix (ICM)}. ICM is a two dimensional symmetric matrix where each row and each column represents one genome. Each position in ICM stores the \textit{Intersection Scores(IS)}. IS is the cardinality number of a core genes which comes from intersecting one genome with other ones. Maximum cardinality results to select two genomes with their maximum core. Mathematically speaking, if we have an $n \times n$ matrix where $n$
-is the number of genomes in local database, then lets consider:\\
+To extract core genes, we iteratively collect the maximum number of
+common genes between genomes and therefore during this stage
+an \textit{Intersection Core Matrix} (ICM) is built. ICM is a two
+dimensional symmetric matrix where each row and each column correspond
+to one genome. Hence, an element of the matrix stores
+the \textit{Intersection Score} (IS): the cardinality of the core
+genes set obtained by intersecting one genome with another
+one. Maximum cardinality results in selecting the two genomes having
+the maximum score. Mathematically speaking, if we have an $n \times n$
+matrix where $n$ is the number of genomes in local database, then let
+us consider:
\begin{equation}
Score=\max_{i<j}\vert x_i \cap x_j\vert
\label{Eq1}
\end{equation}
-\noindent where $x_i, x_j$ are elements in the matrix. The generation of a new core genes is depending on the cardinality value of intersection scores, we call it \textit{Score}:
-$$\text{New Core} = \begin{cases}
+% TO BE CONTINUED
+
+\noindent where $x_i, x_j$ are elements in the matrix. The generation of
+a new core genes is depending on the cardinality value of intersection
+scores, we call it \textit{Score}:
+$$
+\text{New Core} =
+\begin{cases}
\text{Ignored} & \text{if $\textit{Score}=0$;} \\
\text{new Core id} & \text{if $\textit{Score}>0$.}
-\end{cases}$$
-
-if $\textit{Score}=0$ then we have \textit{disjoint relation} \emph{i.e.}, no common genes between two genomes. In this case the system ignores the genome that annul the core gene size. Otherwise, The system removes these two genomes from ICM and add new core genome with a \textit{coreID} of them to ICM for the calculation in next iteration. This process reduces the size of ICM and repeats until all genomes are treated \emph{i.e.} ICM has no more genomes.
-We observe that ICM is very large because of the amount of data that it stores. This results to be time and memory consuming for calculating the intersection scores. To increase the speed of calculations, it is sufficient to only calculate the upper triangle scores. The time complexity for this process after enhancement is thus $O(\frac{n.(n-1)}{2})$. Algorithm \ref{Alg1:ICM} illustrates the construction of the ICM matrix and the extraction of the core genes where \textit{GenomeList}, represents the database where all genomes data are stored. At each iteration, it computes the maximum core genes with its two genomes parents.
+\end{cases}
+$$
+
+if $\textit{Score}=0$ then we have \textit{disjoint
+relation} \emph{i.e.}, no common genes between two genomes. In this
+case the system ignores the genome that annul the core gene
+size. Otherwise, The system removes these two genomes from ICM and add
+new core genome with a \textit{coreID} of them to ICM for the
+calculation in next iteration. This process reduces the size of ICM
+and repeats until all genomes are treated \emph{i.e.} ICM has no more
+genomes. We observe that ICM is very large because of the amount of
+data that it stores. This results to be time and memory consuming for
+calculating the intersection scores. To increase the speed of
+calculations, it is sufficient to only calculate the upper triangle
+scores. The time complexity for this process after enhancement is thus
+$O(\frac{n.(n-1)}{2})$. Algorithm \ref{Alg1:ICM} illustrates the
+construction of the ICM matrix and the extraction of the core genes
+where \textit{GenomeList}, represents the database where all genomes
+data are stored. At each iteration, it computes the maximum core genes
+with its two genomes parents.
\begin{algorithm}[H]
\caption{Extract Maximum Intersection Score}
\end{algorithm}
\subsection{Features Visualization}
-The goal here is to visualize results by building a tree of evolution. All core genes generated with their genes are very important information in the tree, because they can be viewed as an ancestor information for two genomes or more. Further more, each node in the tree represents one chloroplast genome or one predicted core which named under the title of \textit{(Genes count:Family name\_Scientific names\_Accession number)}, Edges represent the number of lost genes from each leaf genome or from an intermediate core genes. The number of lost genes here can represent an important factor for evolution: it represents how much is the lost of genes from the species belongs to same or different families. By the principle of classification, a small number of gene lost among species indicates that those species are close to each other and belong to same family, while big genes lost means that we have an evolutionary relationship between species from different families. To see the picture clearly, Phylogenetic tree is an evolutionary tree generated also by the system. Generating this tree is based on the distances among genes sequences. There are many resources to build such tree (for example: PHYML\cite{guindon2005phyml}, RAxML{\cite{stamatakis2008raxml,stamatakis2005raxml}, BioNJ , and TNT\cite{goloboff2008tnt}}. We consider to use RAxML\cite{stamatakis2008raxml,stamatakis2005raxml} because it is fast and accurate for build large trees for large count of genomes sequences. The procedure of constructing phylogenetic tree stated in the following steps:
+
+The goal is to visualize results by building a tree of evolution. All
+core genes generated represent important information in the tree,
+because they provide information about the ancestors of two or more
+genomes. Each node in the tree represents one chloroplast genome or
+one predicted core called \textit{(Genes count:Family name\_Scientific
+names\_Accession number)}, while an edge is labeled with the number
+genes lost from a leaf genome or an intermediate core gene.
+
+
+The number of lost genes here can represent an important factor
+for evolution: it represents how much is the lost of genes from the
+species belongs to same or different families. By the principle of
+classification, a small number of gene lost among species indicates
+that those species are close to each other and belong to same family,
+while big genes lost means that we have an evolutionary relationship
+between species from different families. To see the picture clearly,
+Phylogenetic tree is an evolutionary tree generated also by the
+system. Generating this tree is based on the distances among genes
+sequences. There are many resources to build such tree (for example:
+PHYML\cite{guindon2005phyml},
+RAxML{\cite{stamatakis2008raxml,stamatakis2005raxml}, BioNJ, and
+TNT\cite{goloboff2008tnt}}.
+
+We use
+RAxML\cite{stamatakis2008raxml,stamatakis2005raxml} because it is fast
+and accurate for build large trees for large count of genomes
+sequences.
+
+The procedure of constructing phylogenetic tree stated in
+the following steps:
\begin{enumerate}
\item Extract gene sequence for all gene in all core genes, store it in database.
\caption{Total overview of the system pipeline}\label{wholesystem}
\end{figure}
+% STOP HERE
+
\section{Implementation}
We implemented four algorithms to extract maximum core genes from large amount of chloroplast genomes. Two algorithms used to extract core genes based on NCBI annotation, and the others based on dogma annotation tool. Evolutionary tree generated as a result from each method implementation. In this section, we will present the four methods, and how they can extract maximum core genes?, and how the developed code will generate the evolutionary tree.
