The trivial and simple idea to construct the core genome is based on the extraction of Genes names (as gene presence or absence). For instant, in this stage neither sequence comparison nor new annotation were made, we just want to extract all gene counts stored in each chloroplast genome then find the intersection core genes based on gene names.\\
\textbf{Step I: pre-processing}\\
-The objective from this step is to organize, solve genes duplications, and generate sets of genes for each genome. The input to the system is a list genomes from NCBI stored as a \textit{.fasta} file that include a collection of Protein coding genes\cite{parra2007cegma,RDogma}(genes that produce protein) with its coding sequences.
-As a preparation step to achieve the set of core genes, we need to translate these genomes using \textit{BioPython} package\cite{chapman2000biopython}, and extracting all information needed to find the core genes. The process starts by converting each genome in fasta format to GenVision\cite{geneVision} format from DNASTAR, and this is not an easy job. The output from this operation is a lists of genes stored in a local database for genomes, their genes names and genes counts. In this stage, we will accumulate some Gene duplications with each genome treated. In other words, duplication in gene name can comes from genes fragments as long as chloroplast DNA sequences. Identical state, which it is the state that each gene present only one time in a genome (i.e Gene has no copy) without considering the position or gene orientation can be reached by filtering the database from redundant gene name. To do this, we have two solutions: first, we made an orthography checking. Orthography checking is used to merge fragments of a gene to be one gene so that we can solve a duplication.
-Second, we convert the list of genes names for each genome (i.e. after orthography check) in the database to be a set of genes names. Mathematically speaking, if $g=\left[g_1,g_2,g_3,g_1,g_3,g_4\right]$ is a list of genes names, by using the definition of a set in mathematics, we will have $set(g)=\{g_1,g_2,g_3,g_4\}$, where each gene represented only ones. With NCBI genomes, we do not have a problem of genes fragments because they already treated it, but there are a problem of genes orthography. This can generate the problem of gene lost in our method and effect in turn the core genes.
+The objective from this step is to organize, solve genes duplications, and generate sets of genes for each genome. The input to the system is a list of genomes from NCBI stored as \textit{.fasta} files that include a collection of Protein coding genes\cite{parra2007cegma,RDogma}(genes that produce protein) with its coding sequences.
+As a preparation step to achieve the set of core genes, we need to translate these genomes using \textit{BioPython} package\cite{chapman2000biopython}, and extracting all information needed to find the core genes. The process starts by converting each genome in fasta format to GenVision\cite{geneVision} format from DNASTAR, and this is not an easy job. The output from this operation is a lists of genes stored in a local database for genomes, their genes names and genes counts. In this stage, we will accumulate some Gene duplications with each genome treated. In other words, duplication in gene name can comes from genes fragments as long as chloroplast DNA sequences. We defines \textit{Identical state} to be the state that each gene present only one time in a genome (i.e Gene has no copy) without considering the position or gene orientation. This state can be reached by filtering the database from redundant gene name. To do this, we have two solutions: first, we made an orthography checking. Orthography checking is used to merge fragments of a gene to form one gene.
+Second, we convert the list of genes names for each genome (i.e. after orthography check) in the database to be a set of genes names. Mathematically speaking, if $G=\left[g_1,g_2,g_3,g_1,g_3,g_4\right]$ is a list of genes names, by using the definition of a set in mathematics, we will have $set(G)=\{g_1,g_2,g_3,g_4\}$, and $|G|=4$ where $|G|$ is the cardinality number of the set $G$ which represent the number of genes in the set. With NCBI genomes, we do not have a problem of genes fragments because they already treated it, but there are a problem of genes orthography. In our method, this can generate the problem of gene lost and effect in turn the core genes.
The whole process of extracting core genome based on genes names and counts among genomes is illustrate in Figure \ref{Fig2}.
\begin{figure}[H]
\end{figure}
\textbf{Step II: Gene Intersection}\\
-The main objective of this step is try to find best core genes from sets of genes in the database. The idea for finding core genes is to collect in each iteration the maximum number of common genes. To do this, the system build an intersection core matrix(ICM). ICM here is a two dimensional symmetric matrix where each row and column represent the list of genomes in the local database. Each position in ICM stores the \textit{intersection scores}. Intersection Score(IS), is the score by intersect in each iteration two sets of genes for two different genomes in the database. Taking maximum score from each row and then taking the maximum of them will result to draw the two genomes with their maximum core. Then, the system remove these two genomes from ICM and add the core of them under a specific name to ICM for the calculation in next iteration. The core genes generated with its set of genes will store in a database for reused in the future. this process repeat until all genomes treated. If maximum intersection core(MIC) equal to 0, the system will avoid this intersection operation and ignore the genome that smash the maximum core genes.\\
-We observe that ICM will result to be very large because of the huge amount of data that it stores. In addition, this will results to be time and memory consuming for calculating the intersection scores by using just genes names. To increase the speed of calculations, we can calculate the upper triangle scores only and exclude diagonal scores. This will reduce whole processing time and memory to half. The time complexity for this process after enhancement changed from $O(n^2)$ to $O((n-1)\log{n})$.\\
+The goal of this step is trying to find maximum core genes from sets of genes in the database. The idea for finding core genes is to collect in each iteration the maximum number of common genes. To do this, the system build an \textit{Intersection core matrix(ICM)}. ICM here is a two dimensional symmetric matrix where each row and column represent a set of genes for one genome in the local database. Each position in ICM stores the \textit{intersection scores}. Intersection Score(IS), is the cardinality number of a core genes comes from intersecting in each iteration the set of genes for one genome with all other gene sets belong to the rest of genomes in the database. Taking maximum cardinality from each row and then taking the maximum of them will result to select the best two genomes with their maximum core. Mathematically speaking, if we have an $mxn$ matrix where $m,n=$number of genomes in database. lets consider $Z=max_{i<j}(\vert x_i \cap x_j\vert)$ where $x_i, x_j$ are sets of row and column elements in a matrix. if $Z=0$ then we have \textit{disjoint relation} (i.e no common genes between to genomes). In this case the system ignore the set of genes that smash the core genes. Otherwise, The system remove these two genomes from ICM and add new core with a coreID of them to ICM for the calculation in next iteration. The partial core genes generated with its set of genes will store in a database for reused for drawing the tree. this process repeat until all genomes treated.
+We observe that ICM will result to be very large because of the huge amount of data that it stores. In addition, this will results to be time and memory consuming for calculating the intersection scores by using just genes names. To increase the speed of calculations, we can calculate the upper triangle scores only and exclude diagonal scores that is why we write $max_{i<j}$. This will reduce whole processing time and memory to half. The time complexity for this process after enhancement changed from $O(n^2)$ to $O((n-1)\log{n})$.\\
+
The Algorithm of construction the matrix and extracting maximum core genes where illustrated in Algorithm \ref{Alg1}. The output from this step is the maximum core genes with its genomes to draw it in a tree.
\begin{algorithm}[H]
\textit{GenomeList} represents the database.\\
\textbf{Step III: Drawing the Tree}\\
-The main objective here is to the results for visualizing a tree of evolution. We use here a directed graph from Dot graphs package\cite{gansner2002drawing} from Graphviz library. The system can produce this tree automatically by using all information available in a database. Core genes generated with their genes can be very important information in the tree, because they can be represented as an ancestor information for two genomes or more. Further more, each node represents a genome or core as \textit{(Genes count:Family name, Scientific names, Accession number)}, Edges represent numbers of lost genes from genomes-core or core-core relationship. The number of lost genes here can be an important factor for evolution, it represents how much lost of genes for the species in same or different families. By the principle of classification, small number of gene lost among species indicate that those species are related together and belong to same family, while big genes lost means that species is far to be in the same family. To see the picture clearly, Phylogenetic tree is an evolutionary tree generated also by the system. Generating this tree is based on the distances among genes sequences. There are many resources to build such tree (for example: PHYML\cite{guindon2005phyml}, RAxML{\cite{stamatakis2008raxml,stamatakis2005raxml}, BioNJ , and TNT\cite{goloboff2008tnt}}. We consider to use RAxML\cite{stamatakis2008raxml,stamatakis2005raxml} to generate this tree.
+The goal here is to visualizing the results by build a tree of evolution. The system can produce this tree automatically by using Dot graphs package\cite{gansner2002drawing} from Graphviz library and all information available in a database. Core genes generated with their genes can be very important information in the tree, because they can viewed as an ancestor information for two genomes or more. Further more, each node represents a genome or core as \textit{(Genes count:Family name, Scientific names, Accession number)}, Edges represent numbers of lost genes from genomes-core or core-core relationship. The number of lost genes here can be an important factor for evolution, it represents how much lost of genes for the species in same or different families. By the principle of classification, small number of gene lost among species indicate that those species are related together and belong to same family, while big genes lost means that species is far to be in the same family. To see the picture clearly, Phylogenetic tree is an evolutionary tree generated also by the system. Generating this tree is based on the distances among genes sequences. There are many resources to build such tree (for example: PHYML\cite{guindon2005phyml}, RAxML{\cite{stamatakis2008raxml,stamatakis2005raxml}, BioNJ , and TNT\cite{goloboff2008tnt}}. We consider to use RAxML\cite{stamatakis2008raxml,stamatakis2005raxml} to generate this tree.
The main drawback from this method is that we can not depending only on genes names because of three causes: first, the genome may have not totally named (This can be found in early versions of NCBI genomes), so we will have some lost sequences. Second, we may have two genes sharing the same name, while their sequences are different. Third, we need to annotate all the genomes.
{to do later}
\subsection{Core genes from Dogma Annotation tool}
-In previous section, extracting core genes based on NCBI annotation caused some lost of genes due to annotation process of NCBI. Annotation tool play an important role for this losts, because it represents the first stage of gene identification. Good annotation tool still be challenged subject. (Genis Parra in 2007) published a paper state that the subject of accurately genomic and/or gene annotation is still an open source problem, even in the best case scenario where any project has all the expert biologists resources to annotate gene structures, the catalogues of genes can still unclear and still less accurate than experts. \cite{Bakke2009} also stated ("Errors in the annotations are routinely deposited in databases such as NCBI and used to validate subsequent annotation errors."). So, good core genes still need good annotation tool. A lot of softwares today’s were developed for extracted core genes for eukaryote and prokaryote organisms such as CEGMA\cite{parra2007cegma}, Coregenes 3.0\cite{zafar2002coregenes}, and Dogma\cite{RDogma}. The most appropriate annotation tool for chloroplast and mitochondrial genomes is Dogma.
+In previous section, extracting core genes based on NCBI annotation caused some lost of genes due to annotation process. Annotation can play an important role for these losts, because it represents the first process of gene identification. Good annotation tool still be challenged subject. (Genis Parra in 2007) published a paper state that the subject of accurately genomic and/or gene annotation is still an open source problem, even in the best case scenario where any project has all the expert biologists resources to annotate gene structures, the catalogues of genes can still unclear and still less accurate than experts. Where \cite{Bakke2009} also state ("Errors in the annotations are routinely deposited in databases such as NCBI and used to validate subsequent annotation errors."). So, good core genes still needs good annotation tool. A lot of software today’s were developed for extracted core genes for eukaryote and prokaryote organisms such as CEGMA\cite{parra2007cegma}, Coregenes 3.0\cite{zafar2002coregenes}, and Dogma\cite{RDogma}. The appropriate annotation tool for plant chloroplast and mitochondrial genomes is Dogma.
\subsubsection{Why Dogma rather than NCBI annotation?}
Dogma is an annotation tool developed in the university of Texas by \cite{RDogma} in 2004. Dogma is an abbreviation of \textit{Dual Organellar GenoMe Annotator}\cite{RDogma} for plant chloroplast and animal mitochondrial genomes.
-It has its own database for translated the genome in all six reading frames by quering the amino acid sequence database using Blast\cite{altschul1990basic}(i.e Blastx) with various parameters, and to identify protein coding genes\cite{parra2007cegma,RDogma} in the input genome based on sequence similarity of genes in Dogma database, further more it can produce the \textit{Transfer RNAs (tRNA)}\cite{RDogma} and the \textit{Ribosomal RNAs (rRNA)}\cite{RDogma} and verifying their start and end positions rather than NCBI annotation tool.
+It has its own database for translated the genome in all six reading frames and query the amino acid sequence database using Blast\cite{altschul1990basic}(i.e Blastx) with various parameters, and to identify protein coding genes\cite{parra2007cegma,RDogma} in the input genome based on sequence similarity of genes in Dogma database. Further more, it can produce the \textit{Transfer RNAs (tRNA)}\cite{RDogma}, and the \textit{Ribosomal RNAs (rRNA)}\cite{RDogma} and verifying their start and end positions rather than NCBI annotation tool.
\subsubsection{Core genes based on Dogma Genes names and counts}
-The method for extracting core genes using Dogma annotation can be more reliable and confidence than NCBI. This is because NCBI annotation carry some annotation errors\cite{Bakke2009}. The method of extracting core genes can summerized in the following steps:\\
-
-\textbf{Step I: Annotate Genomes}\\
-
-In this step, we starts to annotate same chloroplast genome population, this method is done manually for 99 genomes. The output from Dogma annotation website is collection of coding genes file for each genome in GeneVision\cite{geneVision} file format.\\
-
-\textbf{Step two: Genes Defragmentation}
-As a result of the annotation from dogma is coding genes with fragments. It is not considered as gene duplication.
-
-
+The main goal is to get as much as possible the core genes of maximum coding genes. According to \cite{Bakke2009}, annotation method like dogma can give us more reliable coding genes than NCBI. This is because NCBI annotation can carry some annotation and gene identification errors. The general overview of whole process of extraction illustrated in fiqure \ref{dog:Fig1}, the pipeline of extracting core genes can summarize in the following steps:\\
\begin{figure}[H]
-\caption{Core genome based on Dogma Gene Name and count}
\centering
\includegraphics[width=0.7\textwidth]{Dogma_GeneName}
+ \label{dog:Fig1}
+ \caption{Core genome based on Dogma Gene Name and count}
\end{figure}
+\textbf{Step I: Pre-processing}\\
+Pre-processing step represents the key difference between methods of extracting core genes. In figure \ref{dog:Fig1}, The pre-processing step can be divided into two tasks: First, It starts to annotate our genome samples in the form of complete genomes (i.e \textit{Unannotated genomes} from NCBI by using Dogma web tool. The whole annotation process by using dogma website is done manually. The output from the annotation process is considered to be a collection of coding genes file for each genome in the form of GeneVision\cite{geneVision} file format.\\
+Where the second task is to solve gene fragments. Defragment process starts here to solve fragments of coding genes for each genome, this process can avoid gene duplication. All genomes now are fully annotated, their genes were de-fragmented, and genes list and counts identified. These information stored in local database.\\
+
+From these two tasks, we can obtain clearly one copy of coding genes. To ensure that genes produces from dogma annotation process is same as the genes in NCBI. We apply separately a quality check process that align the same gene from dogma and NCBI with respect to a specific threshold.\\
+
+\textbf{Step II: Extraction Core genes}\\
+Extracting core genes will use the same process presented in the section of extracting core genes based on NCBI genes. ICM matrix will be considered by calculating the upper triangular cardinality cores to save time and to find the maximum length of core genes from each iteration see algorithm \ref{Alg1}. From each iteration, two genomes are considered to draw with their maximum cardinality core genes until no genome remain in the database. The key point here is that the intersection genome that smash the core genes in each iteration will be ignored from this competition.\\
+
+\textbf{Step III: Draw the tree}
+To build the tree of evolution for genomes. The algorithm is considered to take from the data base the first coreID generated from step two and draw sequentially all the genomes that create this core. Sometimes, we have a core genome that intersect with another one. This tree also represented as a set of nodes which represent genome names and a set of edges, which represent the number of gene lost from each genome. Phylogenetic tree also considered here by using RAxML{\cite{stamatakis2008raxml,stamatakis2005raxml} based on calculating the distances among core genes in the database.
\subsubsection{Core genome from Dogma gene contents}
[To do Later]
\begin{figure}[H]
-\caption{Core genes based on the comparison of Dogma Genes Sequences}
\centering
\includegraphics[width=0.7\textwidth]{Dogma_GeneContent}
+ \label{dog:Fig2}
+ \caption{Core genes based on the comparison of Dogma Genes Sequences}
\end{figure}
\ No newline at end of file
