integration de remarques de reviewers

[canny.git] / experiments.tex

diff --git a/experiments.tex b/experiments.tex
index 2b1c9e35c87f01a5141af3bcb9ecff5797a54bd6..5ba43dfa16ae206dcee8557c46adc9ef457b7645 100644 (file)
--- a/experiments.tex
+++ b/experiments.tex
@@ -1,161 +1,215 @@
-For whole experiments, the whole set of 10000 images 
+First of all, the whole code of STABYLO can be downloaded
+\footnote{\url{http://members.femto-st.fr/raphael-couturier/stabylo/}}.
+For all the experiments, the whole set of 10,000 images 

 of the BOSS contest~\cite{Boss10} database is taken. 
 In this set, each cover is a $512\times 512$
 grayscale digital image in a RAW format. 
 We restrict experiments to 
 of the BOSS contest~\cite{Boss10} database is taken. 
 In this set, each cover is a $512\times 512$
 grayscale digital image in a RAW format. 
 We restrict experiments to 

-this set of cover images since this paper is more focussed on 
-the methodology than benchmarking.    
-
-
-\subsection{Adaptive Embedding Rate} 
-
-Two strategies have been developed in our scheme, depending on the embedding rate that is either \emph{adaptive} or \emph{fixed}.
-
-In the former the embedding rate depends on the number of edge pixels.
-The higher it is, the larger the message length that can be inserted is.
-Practically, a set of edge pixels is computed according to the 
-Canny algorithm with an high threshold.
-The message length is thus defined to be half of this set cardinality.
-In this strategy, two methods are thus applied to extract bits that 
-are modified. The first one is a direct application of the STC algorithm.
-This method is further referred to as \emph{adaptive+STC}.
-The second one randomly chooses the subset of pixels to modify by 
-applying the BBS PRNG again. This method is denoted \emph{adaptive+sample}.
-Notice that the rate between 
-available bits  and bit message length is always equal to 2.
-This constraint is indeed induced by the fact that the efficiency 
-of the STC algorithm is unsatisfactory under that threshold.
-In our experiments and with the adaptive scheme, 
-the average size of the message that can be embedded is 16,445 bits.
-Its corresponds to an  average payload of 6.35\%. 
+this set of cover images since this paper is more focused on 
+the methodology than on benchmarks.    
+
+We use the matrices $\hat{H}$ 
+generated by the integers given
+in Table~\ref{table:matrices:H}
+as introduced in~\cite{FillerJF11}, since these ones have experimentally 
+be proven to have the strongest modification efficiency.
+For instance if the rate between the size of the message and the size of the 
+cover vector
+is 1/4, each number in $\{81, 95, 107, 121\}$ is translated into a binary number 
+and each one constitutes thus a column of $\hat{H}$. 
+
+\begin{table}
+$$
+\begin{array}{|l|l|}
+\hline
+\textrm{Rate} & \textrm{Matrix generators} \\
+\hline
+{1}/{2} & \{71,109\}\\
+\hline
+{1}/{3} & \{95, 101, 121\}\\
+\hline
+{1}/{4} & \{81, 95, 107, 121\}\\
+\hline
+{1}/{5} & \{75, 95, 97, 105, 117\}\\
+\hline
+{1}/{6} & \{73, 83, 95, 103, 109, 123\}\\
+\hline
+{1}/{7} & \{69, 77, 93, 107, 111, 115, 121\}\\
+\hline
+{1}/{8} & \{69, 79, 81, 89, 93, 99, 107, 119\}\\
+\hline
+{1}/{9} & \{69, 79, 81, 89, 93, 99, 107, 119, 125\}\\
+\hline
+\end{array}
+$$
+\caption{Matrix Generator for $\hat{H}$ in STC}\label{table:matrices:H}
+\end{table}

 
 
 
 

+Our approach is always compared to HUGO, to EAISLSBMR, to WOW and to UNIWARD
+for the two strategies Fixed and Adaptive. 
+For the former one, the payload has been set to 10\%. 
+For the latter one, the Canny parameter $T$ has been set to 3. 
+When $b$ is 7, the average size of the message that can be embedded
+is 16,445 bits, 
+that corresponds to an  average payload of 6.35\%.
+For each cover image the STABYLO's embedding rate with these two parameters 
+is memorized. 
+Next each steganographic scheme is executed to produce the stego content of 
+this cover with respect to this embedding rate.  

 
 
 
 

-In the latter, the embedding rate is defined as a percentage between the 
-number of modified pixels and the length of the bit message.
-This is the classical approach adopted in steganography.
-Practically, the Canny algorithm generates  
-a set of edge pixels related to a threshold that is decreasing until its cardinality
-is sufficient. If the set cardinality is more than twice larger than the 
-bit message length, a STC step is again applied.
-Otherwise, pixels are again randomly chosen with BBS.

 
  
 
 
  
 

-\subsection{Image Quality}
+\subsection{Image quality}\label{sub:quality}

 The visual quality of the STABYLO scheme is evaluated in this section.
 The visual quality of the STABYLO scheme is evaluated in this section.

-For the sake of completeness, four metrics are computed in these experiments: 
+For the sake of completeness, three metrics are computed in these experiments: 

 the Peak Signal to Noise Ratio (PSNR), 
 the PSNR-HVS-M family~\cite{psnrhvsm11}, 
 the Peak Signal to Noise Ratio (PSNR), 
 the PSNR-HVS-M family~\cite{psnrhvsm11}, 

-the BIQI~\cite{MB10}, and 
+and 

 the weighted PSNR (wPSNR)~\cite{DBLP:conf/ih/PereiraVMMP01}.
 The first one is widely used but does not take into
 account the Human Visual System (HVS).
 The other ones have been designed to tackle this problem.
 
 the weighted PSNR (wPSNR)~\cite{DBLP:conf/ih/PereiraVMMP01}.
 The first one is widely used but does not take into
 account the Human Visual System (HVS).
 The other ones have been designed to tackle this problem.
 

+If we apply them on the running example with the Adaptive and STC strategies, 
+the PSNR, PSNR-HVS-M, and wPSNR values are respectively equal to 
+68.39, 79.85, and 89.71 for the stego Lena when $b$ is equal to 7.
+If $b$ is 6, these values are respectively equal to 
+65.43, 77.2, and 89.35.
+
+
+
+

 \begin{table*}
 \begin{center}
 \begin{table*}
 \begin{center}

-\begin{tabular}{|c|c|c||c|c|c|}
+\begin{small}
+\setlength{\tabcolsep}{3pt}
+\begin{tabular}{|c|c|c||c|c|c|c|c|c|c|c|c|c|}

 \hline
 \hline

-Schemes & \multicolumn{3}{|c|}{STABYLO} & \multicolumn{2}{|c|}{HUGO}\\
+Schemes & \multicolumn{4}{|c|}{STABYLO} & \multicolumn{2}{|c|}{HUGO}& \multicolumn{2}{|c|}{EAISLSBMR} & \multicolumn{2}{|c|}{WOW} & \multicolumn{2}{|c|}{UNIWARD}\\

 \hline
 \hline

-Embedding &   \multicolumn{2}{|c||}{Adaptive} & Fixed & \multicolumn{2}{|c|}{Fixed} \\
+Embedding &   Fixed & \multicolumn{3}{|c|}{Adaptive (about 6.35\%)} &  Fixed &Adaptive & Fixed &Adaptive & Fixed &Adaptive & Fixed &Adaptive \\

 \hline
 \hline

-Rate &   + STC &  + sample & 10\% & 10\%&6.35\%\\ 
+Rate &   10\% &  + sample &  +STC(7) & +STC(6) &  10\%&$\approx$6.35\%& 10\%&$\approx$6.35\%& 10\%&$\approx$6.35\%& 10\%&$\approx$6.35\%\\ 

 \hline
 \hline

-PSNR &  66.55 & 63.48  & 61.86  & 64.65 & 67.08  \\ 
+PSNR & 61.86 & 63.48 &  66.55  &  63.7  & 64.65 & {67.08} & 60.8 & 62.9&65.9 & 68.3 & 65.8 & 69.2\\ 

 \hline
 \hline

-PSNR-HVS-M & 78.6 & 75.39 & 72.9 & 76.67 & 79.23 \\ 
+PSNR-HVS-M & 72.9 & 75.39 & 78.6  & 75.5  & 76.67 & {79.6} & 71.8  & 76.0 &
+76.7 & 80.35 & 77.6 & 81.2 \\

 \hline
 \hline

-BIQI & 28.3 & 28.28 & 28.4 & 28.28 & 28.28 \\ 
-\hline
-wPSNR & 86.43& 80.59 & 77.47& 83.03 & 87.8\\ 
+wPSNR & 77.47 & 80.59 & 86.43& 86.28  & 83.03 & {88.6} & 76.7 & 83& 83.8 & 90.4 & 85.2 & 91.9\\ 

 \hline
 \end{tabular}
 \hline
 \end{tabular}

+\end{small}

 \end{center}
 \end{center}

-\caption{Quality Measures of Steganography Approaches\label{table:quality}} 
+\caption{Quality measures of steganography approaches\label{table:quality}}

 \end{table*}
 
 \end{table*}
 

+
+
+Results are summarized in Table~\ref{table:quality}.
+In this table, STC(7) stands for embedding data in the LSB whereas
+in STC(6), data are hidden in the last two  significant bits. 
+
+

 Let us give an interpretation of these experiments.
 Let us give an interpretation of these experiments.

-First of all, the adaptive strategy produces images with lower distortion 
-than the one of images resulting from the 10\% fixed strategy.
+First of all, the Adaptive strategy produces images with lower distortion 
+than the images resulting from the 10\% fixed strategy.

 Numerical results are indeed always greater for the former strategy than 
 Numerical results are indeed always greater for the former strategy than 

-for the latter, except for the BIQI metrics where differences are not really relevant.
-These results are not surprising since the adaptive strategy aims at 
-embedding messages whose length is decided according to an higher threshold
+for the latter one.
+These results are not surprising since the Adaptive strategy aims at 
+embedding messages whose length is decided according to a higher threshold

 into the edge detection.  
 into the edge detection.  

-Let us focus on the quality of HUGO images: with a given fixed 
-embedding rate (10\%), 
-HUGO always produces images whose quality is higher than the STABYLO's one.
-However, our approach nevertheless provides equivalent
-results with the strategy 
-\emph{adaptive+STC} than HUGO with an average embedding rate set to 
-6.35\%. 
-This occurs with a lightweight manner, as motivated in the introduction.      

 
 
 
 

-Let us now compare the STABYLO approach with other edge based steganography
-schemes with respect to the image quality.
-First of all, the Edge Adaptive
-scheme detailed in~\cite{Luo:2010:EAI:1824719.1824720} 
-executed with a 10\% embedding rate 
-has the same PSNR but a lower wPSNR than ours:
-these two metrics are respectively equal to 61.9 and 68.9. 
-Next, both  approaches~\cite{DBLP:journals/eswa/ChenCL10,Chang20101286}
-focus on increasing the payload while the PSNR is acceptable, but do not 
-give quality metrics for fixed embedding rates from a large base of images. 
-Our approach outperforms the former thanks to the introduction of the STC 
-algorithm.
+If we combine Adaptive and STC strategies 
+the STABYLO scheme  provides images whose quality is higher than 
+the EAISLSBMR's one but lower than the quality of high complexity 
+schemes. Notice that the quality of the less respectful scheme (EAILSBMR) 
+is lower than 6\% than the one of the most one.
+
+
+% Let us now compare the STABYLO approach with other edge based steganography
+% approaches, namely~\cite{DBLP:journals/eswa/ChenCL10,Chang20101286}.
+% These two schemes focus on increasing the
+% payload while the PSNR is acceptable, but do not 
+% give quality metrics for fixed embedding rates from a large base of images. 

 
 
 
 
 
 
 
 

-\subsection{Steganalysis}
+\subsection{Steganalysis}\label{sub:steg}

 
 
 
 
 
 

-The quality of our approach has been evaluated through the two 
-AUMP~\cite{Fillatre:2012:ASL:2333143.2333587}
-and Ensemble Classifier~\cite{DBLP:journals/tifs/KodovskyFH12} based steganalysers.
-Both aims at detecting hidden bits in grayscale natural images and are 
-considered as the state of the art of steganalysers in spatial domain~\cite{FK12}.
-The former approach is based on a simplified parametric model of natural images.
-Parameters are firstly estimated and an adaptive Asymptotically Uniformly Most Powerful
-(AUMP) test is designed (theoretically and practically), to check whether
-an image has stego content or not.  
-In the latter, the authors show that the 
-machine learning step, which is often
-implemented as support vector machine,
-can be favorably executed thanks to an ensemble classifier.
+The steganalysis quality of our approach has been evaluated through the % two 
+% AUMP~\cite{Fillatre:2012:ASL:2333143.2333587}
+% and
+Ensemble Classifier~\cite{DBLP:journals/tifs/KodovskyFH12} based steganalyser.
+Its  particularization to spatial domain is 
+considered as state of the art steganalysers.
+Firstly, a space 
+of 686 co-occurrence and Markov features is extracted from the 
+set of cover images and the set of training images. Next a small 
+set of weak classifiers is randomly built,
+each one working on a subspace of all the features.
+The final classifier is constructed by a majority voting 
+between the decisions of these  individual classifiers.

 
 
 
 

+%The former approach is based on a simplified parametric model of natural images.
+% Parameters are firstly estimated and an adaptive Asymptotically Uniformly Most Powerful
+% (AUMP) test is designed (theoretically and practically), to check whether
+% an image has stego content or not.
+% This approach is dedicated to verify whether LSB has been modified or not.
+% , the authors show that the 
+% machine learning step, which is often
+% implemented as a support vector machine,
+% can be favorably executed thanks to an ensemble classifier.
+

 
 \begin{table*}
 \begin{center}
 
 \begin{table*}
 \begin{center}

-%\begin{small}
-\begin{tabular}{|c|c|c|c|c|c|c|c|}
-\hline
-Schemes & \multicolumn{3}{|c|}{STABYLO} & \multicolumn{2}{|c|}{HUGO}& \multicolumn{2}{|c|}{EAIS}\\
+\begin{small}
+\setlength{\tabcolsep}{3pt}
+\begin{tabular}{|c|c|c|c|c|c|c|c|c|c|c|c|c|}

 \hline
 \hline

-Embedding &   \multicolumn{2}{|c|}{Adaptive} & Fixed & \multicolumn{2}{|c|}{Fixed}& \multicolumn{2}{|c|}{Fixed} \\
+Schemes & \multicolumn{4}{|c|}{STABYLO} & \multicolumn{2}{|c|}{HUGO}& \multicolumn{2}{|c|}{EAISLSBMR} &  \multicolumn{2}{|c|}{WOW} &  \multicolumn{2}{|c|}{UNIWARD}\\

 \hline
 \hline

-Rate &   + STC &  + sample & 10\% & 10\%& 6.35\%& 10\%& 6.35\%\\ 
+Embedding & Fixed &   \multicolumn{3}{|c|}{Adaptive (about 6.35\%)}  & Fixed & Adapt. & Fixed & Adapt. & Fixed & Adapt. & Fixed & Adapt. \\

 \hline
 \hline

-AUMP & 0.39  & 0.33  & 0.22     &  0.50 &  0.50 & &  \\
+Rate & 10\% &  + sample &   +STC(7) & +STC(6)   & 10\%& $\approx$6.35\%& 10\%& $\approx$6.35\% & 10\%& $\approx$6.35\%& 10\%& $\approx$6.35\%\\ 

 \hline
 \hline

-Ensemble Classifier & \textbf{0.47} & 0.44 & 0.35     & 0.48 &  0.49  &  0.22  & 0.46 \\
+%AUMP & 0.22 & 0.33 & 0.39  &   0.45    &  0.50 &  0.50 & 0.49 & 0.50 \\
+%\hline
+Ensemble Classifier & 0.35 & 0.44 & 0.47 & 0.47     & 0.48 &  0.49  &  0.43  & 0.47 & 0.48 & 0.49 & 0.46 & 0.49 \\

 
 \hline
 \end{tabular}
 
 \hline
 \end{tabular}

-%\end{small}
+\end{small}

 \end{center}
 \caption{Steganalysing STABYLO\label{table:steganalyse}} 
 \end{table*}
 
 
 \end{center}
 \caption{Steganalysing STABYLO\label{table:steganalyse}} 
 \end{table*}
 
 

-Results show that our approach is more easily detectable than HUGO, which
-is the most secure steganographic tool, as far as we know. However due to its 
-huge number of features integration, it is not lightweight, which justifies 
-in the authors' opinion the consideration of the proposed method.   
-
+Results of average testing errors 
+are summarized in Table~\ref{table:steganalyse}.
+First of all, STC outperforms the sample strategy %for % the two steganalysers
+ as 
+already noticed in the quality analysis presented in the previous section. 
+Next, our approach is more easily detectable than HUGO, 
+WOW and UNIWARD which are the most secure steganographic tool, 
+as far as we know. 
+However by combining Adaptive and STC strategies
+our approach obtains similar results than the ones of these schemes.
+
+Compared to EAILSBMR, we obtain better results when the strategy is 
+Adaptive. 
+However due to its 
+huge number of integration features, it is not lightweight.
+
+All these numerical experiments confirm 
+the objective presented in the motivations:
+providing an efficient steganography approach in a lightweight manner.