Updated figures and extract core section

diff --git a/Dogma_GeneName.png b/Dogma_GeneName.png
index 2d32a4ef3d53113b303924abed098805c153b248..ec40d2c64d945ed5aac46450b9e66f6502675488 100644 (file)
Binary files a/Dogma_GeneName.png and b/Dogma_GeneName.png differ

diff --git a/Dogma_geneextraction.png b/Dogma_geneextraction.png
index e27f24b0d65a285786f2f996f49b84e134950ab0..06b06787a2ca9c90736d3f201aa1d5a6e71b6dec 100644 (file)
Binary files a/Dogma_geneextraction.png and b/Dogma_geneextraction.png differ

diff --git a/NCBI_GeneName.png b/NCBI_GeneName.png
index c591dc8f5bc579010221035f2020dcd3bbff9cdd..54e2e0f8e5ecafb3e7d1390f39cbee3e901ddf31 100644 (file)
Binary files a/NCBI_GeneName.png and b/NCBI_GeneName.png differ

diff --git a/NCBI_geneextraction.png b/NCBI_geneextraction.png
index 575a3c544b23b39dbdf00e8572ab76396a31b6a7..ca5dae9b86d621ed31002913e6a93cd1d9f09d3d 100644 (file)
Binary files a/NCBI_geneextraction.png and b/NCBI_geneextraction.png differ

diff --git a/Whole_system.png b/Whole_system.png
new file mode 100644 (file)
index 0000000..0e8e31b
Binary files /dev/null and b/Whole_system.png differ

diff --git a/annotated.tex b/annotated.tex
index cfe54c9228b69c0a947979fcaec0faab8433b152..f28d7423598499d6b7653756f021164d35d5a054 100644 (file)
--- a/annotated.tex
+++ b/annotated.tex
@@ -74,24 +74,33 @@ Where the second task is to solve gene fragments. Defragment process starts imme
 
 From these two tasks, we can obtain clearly one copy of coding genes. To ensure that genes produced from dogma annotation process is same as the genes in NCBI. We apply in parrallel a quality checking process that align each gene from dogma and NCBI with respect to a specific threshold.\\
 
 
 From these two tasks, we can obtain clearly one copy of coding genes. To ensure that genes produced from dogma annotation process is same as the genes in NCBI. We apply in parrallel a quality checking process that align each gene from dogma and NCBI with respect to a specific threshold.\\
 

-\subsection{Extract Gene Features}
-The goal of this step is trying to find maximum core genes from sets of genes (\textit{Vectors}) where stored in the local database from the annotation process. The key of finding core genes is to collect from each iteration of genes comparisons the maximum number of common genes. To do so, the system build an \textit{Intersection core matrix(ICM)}. ICM here is a two dimensional symmetric matrix (considered as a vector space) where each row and column represent a vector for one genome. Each position in ICM stores the \textit{intersection scores}. Intersection Score(IS) is the cardinality number of a core genes comes from intersecting one vector with other vectors in vector space. Taking maximum cardinality from each row and then take the maximum of them will result to select the best two genomes with their maximum core. Mathematically speaking, if we have an $m \times n$ vector space matrix where $m=n=$number of vectors in local database, then lets consider:\\
+\subsection{Extract Core Genes}
+The goal of this step is trying to extract maximum core genes from sets of genes (\textit{Vectors}) in the local database. The methodology of finding core genes is dividing to three methods: \\
+
+The hypothesis in first method is based on extracting core genes by finding common genes among chloroplast genomes based on extracting gene feature (i.e Gene names, genes counts). Genomes vary in genes counts according to the method of annotation used, so that extracting maximum core genes can be done by constructing Intersection Core Matrix (\textit{ICM}).\\
+While the hypothesis of second method is based on comparing the sequence of  reference genes of one annotated genome with other unannotated genomes sequences in Blast database, by using Blastn\cite{Sayers01012011} (nucleotide sequence alignment tool from NCBI). The last method, is based on merge all genes from NCBI and Dogma annotation, then apply a sequence similarity base method (Quality Control test) using Needle-man Wunch algorithm to predict a new genomes. Using predicted genomes to extract core genes using previous methods. Figure \ref{wholesystem}, illustrate the whole system operations.
+
+\begin{figure}[H]
+  \centering
+    \includegraphics[width=0.7\textwidth]{Whole_system}
+    \caption{Total overview of the system pipline}\label{wholesystem}
+\end{figure}
+
+In the first method, the idea is to collect from each iteration the maximum number of common genes. To do so, the system build an \textit{Intersection core matrix(ICM)}. ICM here is a two dimensional symmetric matrix (considered as a vector space) where each row and column represent a vector for one genome. Each position in ICM stores the \textit{intersection scores}. Intersection Score(IS) is the cardinality number of a core genes comes from intersecting one vector with other vectors in vector space. Taking maximum cardinality from each row and then take the maximum of them will result to select the maximum cardinality in the vector space. Maximum cardinality results to select two genomes with their maximum core. Mathematically speaking, if we have an $n \times m$ vector space matrix where $n=m=\text{number of vectors in local database}$, then lets consider:\\

 
 \begin{equation}
 Score=\max_{i<j}\vert x_i \cap x_j\vert
 \label{Eq1}
 \end{equation}\\
 
 
 \begin{equation}
 Score=\max_{i<j}\vert x_i \cap x_j\vert
 \label{Eq1}
 \end{equation}\\
 

-Where $x_i, x_j$ are vectors. To constract new core genes, we look on the value of intersecting two vectors, we call it $Score$:\\
+Where $x_i, x_j$ are vectors in the matrix. Generate new core genes is depending on the value of intersecting two vectors, we call it $Score$:\\

 $$\text{New Core} = \begin{cases} 
 \text{Ignored} & \text{if $Score=0$;} \\
 \text{new Core id} & \text{if $Score>0$.}
 \end{cases}$$\\
 
 $$\text{New Core} = \begin{cases} 
 \text{Ignored} & \text{if $Score=0$;} \\
 \text{new Core id} & \text{if $Score>0$.}
 \end{cases}$$\\
 

-if $Score=0$ then we have \textit{disjoint relation} (i.e no common genes between two genomes). In this case the system ignore the vector that smash the core genes. Otherwise, The system remove these two vectors from ICM and add new core vector with a \textit{coreID} of them to ICM for the calculation in next iteration. The partial core vectors generated with its values will store in the local database for reused to draw the tree. this process repeat until all vectors treated.   
-We observe that ICM will result to be very large because of the huge amount of data that it stores. In addition, this will results to be time and memory consuming for calculating the intersection scores by using just genes names. To increase the speed of calculations, we can calculate the upper triangle scores only and exclude diagonal scores. This will reduce whole processing time and memory to half. The time complexity for this process after enhancement changed from $O(n^2-n)$ to $O(\frac{(n-1).n}{2})$.\\
-
-The Algorithm of construction the vector matrix and extracting the vector of maximum core genes where illustrated in Algorithm \ref{Alg1}. The output from this step is the maximum core vector with its two vectors to draw it in a tree.  
+if $Score=0$ then we have \textit{disjoint relation} (i.e no common genes between two genomes). In this case the system ignore the vector that smash the core genes. Otherwise, The system will remove these two vectors from ICM and add new core vector with a \textit{coreID} of them to ICM for the calculation in next iteration. The partial core vectors generated with its values will store in the local database for reused to draw the tree. this process repeat until all vectors treated.   
+We observe that ICM will result to be very large because of the huge amount of data that it stores. In addition, this will results to be time and memory consuming for calculating the intersection scores by using just genes names. To increase the speed of calculations, we can calculate the upper triangle scores only and exclude diagonal scores. This will reduce whole processing time and memory to half. The time complexity for this process after enhancement changed from $O(n^2-n)$ to $O(\frac{(n-1).n}{2})$. The Algorithm of construction the vector matrix and extracting the vector of maximum core genes where illustrated in Algorithm \ref{Alg1}. The output from this step is the maximum core vector with its two vectors to draw it in a tree.\\

 
 \begin{algorithm}[H]
 \caption{Extract Maximum Intersection Score}
 
 \begin{algorithm}[H]
 \caption{Extract Maximum Intersection Score}


@@ -123,16 +132,21 @@ The Algorithm of construction the vector matrix and extracting the vector of max
 \RETURN $max(B1)$
 \end{algorithmic}
 \end{algorithm}
 \RETURN $max(B1)$
 \end{algorithmic}
 \end{algorithm}

-

 \textit{GenomeList} represents the local database.\\
 
 \textit{GenomeList} represents the local database.\\
 

-\subsection{Genomes Relationships}
+In second Method, due to the number of annotated genomes, annotate each genome can be very exhausted task specially with Dogma, because dogma offer a web tool for annotation, so that, each genome must annotate using this web tool. This operation need to do manually. We prefer to recover this problem by choosing one reference chloroplast and querying each reference gene by using \textit{Blastn} to examin its existance in remaining unannotated genomes in blast database. collect all match genomes from each gene hits, to satisfy the hypothesis "the gene who exists in maximum number of genomes also exist in a core genes". In addition, we can also extract the maximum core genes by examine how many genes present with each genome?.\\
+
+The hypothesis in last method state: we can predict the best annotated genome by merge the annotated genomes from NCBI and dogma based on the quality of genes names and sequences. To generate all quality genes of each genome. the hypothesis state: Any gene will be in predicted genome if and only if the annotated genes between NCBI and Dogma pass a specific threshold of\textit{quality control test}. To accept the quality test, we applied Needle-man Wunch algorithm to compare two gene sequences with respect to pass a threshold. If the alignment score pass this threshold, then the gene will be in the predicted genome, else the gene will be ignored. After predicting all genomes, one of previous two methods can be applied to extract core genes.          
+        
+\subsection{Visualizing Relationships}

 The goal here is to visualizing the results by build a tree of evolution. The system can produce this tree automatically by using Dot graphs package\cite{gansner2002drawing} from Graphviz library and all information available in a database. Core genes generated with their genes can be very important information in the tree, because they can viewed as an ancestor information for two genomes or more. Further more, each node represents a genome or core as \textit{(Genes count:Family name, Scientific names, Accession number)}, Edges represent numbers of lost genes from genomes-core or core-core relationship. The number of lost genes here can represent an important factor for evolution, it represents how much lost of genes for the species in same or different families. By the principle of classification, small number of gene lost among species indicate that those species are close to each other and belong to same family, while big genes lost means that species is far to be in the same family. To see the picture clearly, Phylogenetic tree is an evolutionary tree generated also by the system. Generating this tree is based on the distances among genes sequences. There are many resources to build such tree (for example: PHYML\cite{guindon2005phyml}, RAxML{\cite{stamatakis2008raxml,stamatakis2005raxml}, BioNJ , and TNT\cite{goloboff2008tnt}}. We consider to use RAxML\cite{stamatakis2008raxml,stamatakis2005raxml} to generate this tree.
 
 \section{Implementation}
 We implemented four algorithms to extract maximum core genes from large amount of chloroplast genomes. Two algorithms used to extract core genes based on NCBI annotation, and the others based on dogma annotation tool. Evolutionary tree generated as a result from each method implementation. In this section, we will present the four methods, and how they can extract maximum core genes?, and how the developed code will generate the evolutionary tree.
 
 The goal here is to visualizing the results by build a tree of evolution. The system can produce this tree automatically by using Dot graphs package\cite{gansner2002drawing} from Graphviz library and all information available in a database. Core genes generated with their genes can be very important information in the tree, because they can viewed as an ancestor information for two genomes or more. Further more, each node represents a genome or core as \textit{(Genes count:Family name, Scientific names, Accession number)}, Edges represent numbers of lost genes from genomes-core or core-core relationship. The number of lost genes here can represent an important factor for evolution, it represents how much lost of genes for the species in same or different families. By the principle of classification, small number of gene lost among species indicate that those species are close to each other and belong to same family, while big genes lost means that species is far to be in the same family. To see the picture clearly, Phylogenetic tree is an evolutionary tree generated also by the system. Generating this tree is based on the distances among genes sequences. There are many resources to build such tree (for example: PHYML\cite{guindon2005phyml}, RAxML{\cite{stamatakis2008raxml,stamatakis2005raxml}, BioNJ , and TNT\cite{goloboff2008tnt}}. We consider to use RAxML\cite{stamatakis2008raxml,stamatakis2005raxml} to generate this tree.
 
 \section{Implementation}
 We implemented four algorithms to extract maximum core genes from large amount of chloroplast genomes. Two algorithms used to extract core genes based on NCBI annotation, and the others based on dogma annotation tool. Evolutionary tree generated as a result from each method implementation. In this section, we will present the four methods, and how they can extract maximum core genes?, and how the developed code will generate the evolutionary tree.
 

-\subsection{Extract Core genes based on NCBI Gene names and counts}
+\subsection{Extract Core Genes based on Gene Contents}
+
+\subsubsection{Core Genes based on NCBI Annotation}

 The first idea to construct the core genome is based on the extraction of Genes names (as gene presence or absence). For instant, in this stage neither sequence comparison nor new annotation were made, we just want to extract all genes with counts stored in each chloroplast genome, then find the intersection core genes based on gene names. \\
 The pipeline of extracting core genes can summarize in the following steps:\\
 First, we apply the genome annotation method using NCBI annotation tool. Genome quality check can be used in this step to ensure that genomes pass some quality condition. Then, the system lunch annotation process using NCBI to extract code genes (i.e \textit{exons}) and solve gene fragments.  From NCBI, we did not observe any problem with genes fragments, but there are a problem of genes orthography (e.g two different genes sequences with same gene name). After we obtain all annotated genomes from NCBI to the local database, the code will then automatically will generate GenVision\cite{geneVision} file format to lunch the second step to extract coding genes names and counts. The competition will start by building intersection matrix to intersect genomes vectors in the local database with the others. New core vector for two leaf vectors will generate and a specific \textit{CoreId} will assign to it. an evolutionary tree will take place by using all data generated from step 1 and 2. The tree will also display the amount of genes lost from each intersection iteration. A specific excel file will be generated that store all the data in local database. The whole operation illstrate in Figure \ref{NCBI:geneextraction}.
 The first idea to construct the core genome is based on the extraction of Genes names (as gene presence or absence). For instant, in this stage neither sequence comparison nor new annotation were made, we just want to extract all genes with counts stored in each chloroplast genome, then find the intersection core genes based on gene names. \\
 The pipeline of extracting core genes can summarize in the following steps:\\
 First, we apply the genome annotation method using NCBI annotation tool. Genome quality check can be used in this step to ensure that genomes pass some quality condition. Then, the system lunch annotation process using NCBI to extract code genes (i.e \textit{exons}) and solve gene fragments.  From NCBI, we did not observe any problem with genes fragments, but there are a problem of genes orthography (e.g two different genes sequences with same gene name). After we obtain all annotated genomes from NCBI to the local database, the code will then automatically will generate GenVision\cite{geneVision} file format to lunch the second step to extract coding genes names and counts. The competition will start by building intersection matrix to intersect genomes vectors in the local database with the others. New core vector for two leaf vectors will generate and a specific \textit{CoreId} will assign to it. an evolutionary tree will take place by using all data generated from step 1 and 2. The tree will also display the amount of genes lost from each intersection iteration. A specific excel file will be generated that store all the data in local database. The whole operation illstrate in Figure \ref{NCBI:geneextraction}.


@@ -143,7 +157,7 @@ First, we apply the genome annotation method using NCBI annotation tool. Genome
     \caption{Extract core genes based on NCBI gene names and counts}\label{NCBI:geneextraction}
 \end{figure}
 
     \caption{Extract core genes based on NCBI gene names and counts}\label{NCBI:geneextraction}
 \end{figure}
 

-\subsection{Extract Core genes based on Dogma Gene names and counts}  
+\subsubsection{Core Genes based on Dogma Annotation}  

 The main goal is to get as much as possible the core genes of maximum coding genes names. According to NCBI annotation problem based on \cite{Bakke2009}, annotation method like dogma can give us more reliable coding genes than NCBI. This is because NCBI annotation can carry some annotation and gene identification errors. The general overview of whole process of extraction illustrated in figure \ref{dog:Annotation}. From this figure, the pipeline of extracting core genes can summarize in the following steps:\\
 First, we apply the genome annotation method using Dogma annotation tool. Genome quality check can be used in this step to ensure that genomes pass some quality condition. Then, the system lunch annotation process using Dogma to extract code genes (i.e \textit{exons}) and solve gene fragments. The key difference here is that dogma can generate in addition transfer RNA and ribosomal RNA. As a result from annotation process with dogma is genomes files in GenVision\cite{geneVision} file format, the code will lunch genes de-fragments process to avoid genes duplications. little problems of genes orthography (e.g two different genes sequences with same gene name) where exists. After we obtain all annotated genomes from dogma, we store it in the local database. The code will then automatically lunch the second step to extract coding genes names and counts. The competition will start by building intersection matrix to intersect genomes vectors in the local database with the others. New core vector for two leaf vectors will generate and a specific \textit{CoreId} will assign to it. an evolutionary tree will take place by using all data generated from step 1 and 2. The tree will also display the amount of genes lost from each intersection iteration. A specific excel file will be generated that store all the data in local database. The whole operation illstrate in Figure \ref{dogma:geneextraction}.
 
 The main goal is to get as much as possible the core genes of maximum coding genes names. According to NCBI annotation problem based on \cite{Bakke2009}, annotation method like dogma can give us more reliable coding genes than NCBI. This is because NCBI annotation can carry some annotation and gene identification errors. The general overview of whole process of extraction illustrated in figure \ref{dog:Annotation}. From this figure, the pipeline of extracting core genes can summarize in the following steps:\\
 First, we apply the genome annotation method using Dogma annotation tool. Genome quality check can be used in this step to ensure that genomes pass some quality condition. Then, the system lunch annotation process using Dogma to extract code genes (i.e \textit{exons}) and solve gene fragments. The key difference here is that dogma can generate in addition transfer RNA and ribosomal RNA. As a result from annotation process with dogma is genomes files in GenVision\cite{geneVision} file format, the code will lunch genes de-fragments process to avoid genes duplications. little problems of genes orthography (e.g two different genes sequences with same gene name) where exists. After we obtain all annotated genomes from dogma, we store it in the local database. The code will then automatically lunch the second step to extract coding genes names and counts. The competition will start by building intersection matrix to intersect genomes vectors in the local database with the others. New core vector for two leaf vectors will generate and a specific \textit{CoreId} will assign to it. an evolutionary tree will take place by using all data generated from step 1 and 2. The tree will also display the amount of genes lost from each intersection iteration. A specific excel file will be generated that store all the data in local database. The whole operation illstrate in Figure \ref{dogma:geneextraction}.
 


@@ -155,8 +169,10 @@ First, we apply the genome annotation method using Dogma annotation tool. Genome
 
 The main drawback from the method of extracting core genes based on gene names and counts is that we can not depending only on genes names because of three causes: first, the genome may have not totally named (This can be found in early versions of NCBI genomes), so we will have some lost sequences. Second, we may have two genes sharing the same name, while their sequences are different. Third, we need to annotate all the genomes.
 
 
 The main drawback from the method of extracting core genes based on gene names and counts is that we can not depending only on genes names because of three causes: first, the genome may have not totally named (This can be found in early versions of NCBI genomes), so we will have some lost sequences. Second, we may have two genes sharing the same name, while their sequences are different. Third, we need to annotate all the genomes.
 

-\subsubsection{Extracting Core genome from NCBI gene contents}
-{to do later}
+\subsection{Extract Core Genes based on Genes Sequences}
+
+\subsubsection{Core Genes from NCBI Annotation}

 
 

+\subsubsection{Core Genes from Dogma Annotation}

 
 

-\subsubsection{Core genes based on Dogma Genes names and counts}
+\subsection{Extract Core Genes based on Gene Quality Control}

diff --git a/generalView.png b/generalView.png
index 185268d94b57575c38edf5fb3658d6a61446e1c9..9b0a5ad592692dc82c74aaf43f54573591a3a41d 100644 (file)
Binary files a/generalView.png and b/generalView.png differ