<html><head><meta name="color-scheme" content="light dark"></head><body><pre style="word-wrap: break-word; white-space: pre-wrap;">X-Git-Url: https://bilbo.iut-bm.univ-fcomte.fr/and/gitweb/canny.git/blobdiff_plain/3c3e9a12f31ac42d0c3e7eb1be25aeafb7f7b3db..ff62ab9a78e83759f5e40e3b7d422fddeb45115d:/experiments.tex?ds=inline

diff --git a/experiments.tex b/experiments.tex
index aed0505..6d65b81 100644
--- a/experiments.tex
+++ b/experiments.tex
@@ -1,173 +1,228 @@
-For whole experiments, the whole set of 10000 images 
+First of all, the whole code of STABYLO can be downloaded
+\footnote{\url{http://http://members.femto-st.fr/jf-couchot/en/stabylo}}.
+For all the experiments, the whole set of 10,000 images 
 of the BOSS contest~\cite{Boss10} database is taken. 
 In this set, each cover is a $512\times 512$
 grayscale digital image in a RAW format. 
 We restrict experiments to 
-this set of cover images since this paper is more focussed on 
-the methodology than benchmarking.    
-Our approach is always compared to Hugo~\cite{DBLP:conf/ih/PevnyFB10}
-and to EAISLSBMR~\cite{Luo:2010:EAI:1824719.1824720}.
-
-
-
-
-
-\subsection{Adaptive Embedding Rate} 
-Two strategies have been developed in our scheme, 
-depending on the embedding rate that is either \emph{adaptive} or \emph{fixed}.
-
-In the former the embedding rate depends on the number of edge pixels.
-The higher it is, the larger the message length that can be inserted is.
-Practically, a set of edge pixels is computed according to the 
-Canny algorithm with an high threshold.
-The message length is thus defined to be half of this set cardinality.
-In this strategy, two methods are thus applied to extract bits that 
-are modified. The first one is a direct application of the STC algorithm.
-This method is further referred to as \emph{adaptive+STC}.
-The second one randomly chooses the subset of pixels to modify by 
-applying the BBS PRNG again. This method is denoted \emph{adaptive+sample}.
-Notice that the rate between 
-available bits  and bit message length is always equal to 2.
-This constraint is indeed induced by the fact that the efficiency 
-of the STC algorithm is unsatisfactory under that threshold.
-In our experiments and with the adaptive scheme, 
-the average size of the message that can be embedded is 16,445 bits.
-Its corresponds to an  average payload of 6.35\%. 
-
-
-
-
-In the latter, the embedding rate is defined as a percentage between the 
-number of modified pixels and the length of the bit message.
-This is the classical approach adopted in steganography.
-Practically, the Canny algorithm generates  
-a set of edge pixels related to a threshold that is decreasing until its cardinality
-is sufficient. If the set cardinality is more than twice larger than the 
-bit message length, a STC step is again applied.
-Otherwise, pixels are again randomly chosen with BBS.
-
- 
-
-\subsection{Image Quality}
-The visual quality of the STABYLO scheme is evaluated in this section.
-For the sake of completeness, four metrics are computed in these experiments: 
-the Peak Signal to Noise Ratio (PSNR), 
-the PSNR-HVS-M family~\cite{psnrhvsm11}, 
-%the BIQI~\cite{MB10}, 
-and 
-the weighted PSNR (wPSNR)~\cite{DBLP:conf/ih/PereiraVMMP01}.
-The first one is widely used but does not take into
-account the Human Visual System (HVS).
-The other ones have been designed to tackle this problem.
-
-
-
-
-\begin{table*}
-\begin{center}
-\begin{tabular}{|c|c|c||c|c|c|c|c|}
+this set of cover images since this paper is more focused on 
+the methodology than on benchmarks.    
+
+We use the matrices $\hat{H}$ 
+generated by the integers given
+in Table~\ref{table:matrices:H}
+as introduced in~\cite{FillerJF11}, since these ones have experimentally 
+be proven to have the strongest modification efficiency.
+For instance if the rate between the size of the message and the size of the 
+cover vector
+is 1/4, each number in $\{81, 95, 107, 121\}$ is translated into a binary number 
+and each one constitutes thus a column of $\hat{H}$. 
+
+\begin{table}
+$$
+\begin{array}{|l|l|}
 \hline
-Schemes &amp; \multicolumn{3}{|c|}{STABYLO} &amp; \multicolumn{2}{|c|}{HUGO}&amp; \multicolumn{2}{|c|}{EAISLSBMR} \\
+\textrm{Rate} &amp; \textrm{Matrix generators} \\
 \hline
-Embedding &amp;   \multicolumn{2}{|c||}{Adaptive} &amp; Fixed &amp; \multicolumn{2}{|c|}{Fixed}&amp; \multicolumn{2}{|c|}{Fixed} \\
+{1}/{2} &amp; \{71,109\}\\
 \hline
-Rate &amp;   + STC &amp;  + sample &amp; 10\% &amp; 10\%&amp;6.35\%&amp; 10\%&amp;6.35\%\\ 
+{1}/{3} &amp; \{95, 101, 121\}\\
 \hline
-PSNR &amp;  66.55 (\textbf{-0.8\%}) &amp; 63.48  &amp; 61.86  &amp; 64.65 &amp; {67.08} &amp; 60.8 &amp; 62.9\\ 
+{1}/{4} &amp; \{81, 95, 107, 121\}\\
 \hline
-PSNR-HVS-M &amp; 78.6 (\textbf{-0.8\%})  &amp; 75.39 &amp; 72.9 &amp; 76.67 &amp; {79.23} &amp; 61.3  &amp; 63.4\\ 
-%\hline
-%BIQI &amp; 28.3 &amp; 28.28 &amp; 28.4 &amp; 28.28 &amp; 28.28 &amp; 28.2 &amp; 28.2\\ 
+{1}/{5} &amp; \{75, 95, 97, 105, 117\}\\
 \hline
-wPSNR &amp; 86.43(\textbf{-1.6\%}) &amp; 80.59 &amp; 77.47&amp; 83.03 &amp; {87.8} &amp; &amp; 80.6\\ 
+{1}/{6} &amp; \{73, 83, 95, 103, 109, 123\}\\
 \hline
-\end{tabular}
-\end{center}
-\caption{Quality Measures of Steganography Approaches\label{table:quality}}
-\label{table:quality}
-\end{table*}
-
-
-
-Results are summarized into the Table~\ref{table:quality}.
-Let us give an interpretation of these first experiments.
-First of all, the adaptive strategy produces images with lower distortion 
-than the one of images resulting from the 10\% fixed strategy.
-Numerical results are indeed always greater for the former strategy than 
-for the latter.
-These results are not surprising since the adaptive strategy aims at 
-embedding messages whose length is decided according to an higher threshold
-into the edge detection.  
-Let us focus on the quality of HUGO images: with a given fixed 
-embedding rate (10\%), 
-HUGO always produces images whose quality is higher than the STABYLO's one.
-However our appraoch always outperforms EAISLSBMR since this one may modify 
-the two least significant bits whereas STABYLO only alter LSB.
-
-If we combine \emph{adaptive} and \emph{STC} strategies 
-(which leads to an average embedding rate equal to 6.35\%)
-our approach  provides equivalent metrics than HUGO.
-The quality variance between HUGO and STABYLO for these parameters
-is given in bold font. It is always close to 1\% which confirms 
-the objective presented in the motivations:
-providing an efficient steganography approach with a lightweight manner.
-
-
-Let us now compare the STABYLO approach with other edge based steganography
-approaches, namely~\cite{DBLP:journals/eswa/ChenCL10,Chang20101286}.
-These two schemes focus on increasing the
-payload while the PSNR is acceptable, but do not 
-give quality metrics for fixed embedding rates from a large base of images. 
-
+{1}/{7} &amp; \{69, 77, 93, 107, 111, 115, 121\}\\
+\hline
+{1}/{8} &amp; \{69, 79, 81, 89, 93, 99, 107, 119\}\\
+\hline
+{1}/{9} &amp; \{69, 79, 81, 89, 93, 99, 107, 119, 125\}\\
+\hline
+\end{array}
+$$
+\caption{Matrix Generator for $\hat{H}$ in STC}\label{table:matrices:H}
+\end{table}
 
 
+Our approach is always compared to HUGO, to EAISLSBMR, to WOW and to UNIWARD
+for the two strategies Fixed and Adaptive. 
+For the former one, the payload has been set to 10\%. 
+For the latter one, the Canny parameter $T$ has been set to 3. 
+When $b$ is 7, the average size of the message that can be embedded
+is 16,445 bits, 
+that corresponds to an  average payload of 6.35\%.
+For each cover image the STABYLO's embedding rate with these two parameters 
+is memorized. 
+Next each steganographic scheme is executed to produce the stego content of 
+this cover with respect to this embedding rate.  
 
-\subsection{Steganalysis}
 
 
+ 
 
-The quality of our approach has been evaluated through the two 
-AUMP~\cite{Fillatre:2012:ASL:2333143.2333587}
-and Ensemble Classifier~\cite{DBLP:journals/tifs/KodovskyFH12} based steganalysers.
-Both aims at detecting hidden bits in grayscale natural images and are 
-considered as the state of the art of steganalysers in spatial domain~\cite{FK12}.
-The former approach is based on a simplified parametric model of natural images.
-Parameters are firstly estimated and an adaptive Asymptotically Uniformly Most Powerful
-(AUMP) test is designed (theoretically and practically), to check whether
-an image has stego content or not.
-This approach is dedicated to verify whether LSB has been modified or not.
-In the latter, the authors show that the 
-machine learning step, which is often
-implemented as support vector machine,
-can be favorably executed thanks to an ensemble classifier.
+% \subsection{Image quality}\label{sub:quality}
+% The visual quality of the STABYLO scheme is evaluated in this section.
+% For the sake of completeness, three metrics are computed in these experiments: 
+% the Peak Signal to Noise Ratio (PSNR), 
+% the PSNR-HVS-M family~\cite{psnrhvsm11}, 
+% and 
+% the weighted PSNR (wPSNR)~\cite{DBLP:conf/ih/PereiraVMMP01}.
+% The first one is widely used but does not take into
+% account the Human Visual System (HVS).
+% The other ones have been designed to tackle this problem.
+
+% If we apply them on the running example with the Adaptive and STC strategies, 
+% the PSNR, PSNR-HVS-M, and wPSNR values are respectively equal to 
+% 68.39, 79.85, and 89.71 for the stego Lena when $b$ is equal to 7.
+% If $b$ is 6, these values are respectively equal to 
+% 65.43, 77.2, and 89.35.
+
+
+
+
+% \begin{table*}
+% \begin{center}
+% \begin{small}
+% \setlength{\tabcolsep}{3pt}
+% \begin{tabular}{|c|c|c||c|c|c|c|c|c|c|c|c|c|}
+% \hline
+% Schemes &amp; \multicolumn{4}{|c|}{STABYLO} &amp; \multicolumn{2}{|c|}{HUGO}&amp; \multicolumn{2}{|c|}{EAISLSBMR} &amp; \multicolumn{2}{|c|}{WOW} &amp; \multicolumn{2}{|c|}{UNIWARD}\\
+% \hline
+% Embedding &amp;   Fixed &amp; \multicolumn{3}{|c|}{Adaptive (about 6.35\%)} &amp;  Fixed &amp;Adaptive &amp; Fixed &amp;Adaptive &amp; Fixed &amp;Adaptive &amp; Fixed &amp;Adaptive \\
+% \hline
+% Rate &amp;   10\% &amp;  + sample &amp;  +STC(7) &amp; +STC(6) &amp;  10\%&amp;$\approx$6.35\%&amp; 10\%&amp;$\approx$6.35\%&amp; 10\%&amp;$\approx$6.35\%&amp; 10\%&amp;$\approx$6.35\%\\ 
+% \hline
+% PSNR &amp; 61.86 &amp; 63.48 &amp;  66.55  &amp;  63.7  &amp; 64.65 &amp; {67.08} &amp; 60.8 &amp; 62.9&amp;65.9 &amp; 68.3 &amp; 65.8 &amp; 69.2\\ 
+% \hline
+% PSNR-HVS-M &amp; 72.9 &amp; 75.39 &amp; 78.6  &amp; 75.5  &amp; 76.67 &amp; {79.6} &amp; 71.8  &amp; 76.0 &amp;
+% 76.7 &amp; 80.35 &amp; 77.6 &amp; 81.2 \\
+% \hline
+% wPSNR &amp; 77.47 &amp; 80.59 &amp; 86.43&amp; 86.28  &amp; 83.03 &amp; {88.6} &amp; 76.7 &amp; 83&amp; 83.8 &amp; 90.4 &amp; 85.2 &amp; 91.9\\ 
+% \hline
+% \end{tabular}
+% \end{small}
+% \end{center}
+% \caption{Quality measures of steganography approaches\label{table:quality}}
+% \end{table*}
+
+
+
+% Results are summarized in Table~\ref{table:quality}.
+% In this table, STC(7) stands for embedding data in the LSB whereas
+% in STC(6), data are hidden in the last two  significant bits. 
+
+
+% Let us give an interpretation of these experiments.
+% First of all, the Adaptive strategy produces images with lower distortion 
+% than the images resulting from the 10\% fixed strategy.
+% Numerical results are indeed always greater for the former strategy than 
+% for the latter one.
+% These results are not surprising since the Adaptive strategy aims at 
+% embedding messages whose length is decided according to a higher threshold
+% into the edge detection.  
+
+
+% If we combine Adaptive and STC strategies 
+% the STABYLO scheme  provides images whose quality is higher than 
+% the EAISLSBMR's one but lower than the quality of high complexity 
+% schemes. Notice that the quality of the less respectful scheme (EAILSBMR) 
+% is lower than 6\% than the one of the most one.
+
+
+% % Let us now compare the STABYLO approach with other edge based steganography
+% % approaches, namely~\cite{DBLP:journals/eswa/ChenCL10,Chang20101286}.
+% % These two schemes focus on increasing the
+% % payload while the PSNR is acceptable, but do not 
+% % give quality metrics for fixed embedding rates from a large base of images. 
+
+
+
+
+\subsection{Steganalysis}\label{sub:steg}
+
+
+
+The steganalysis quality of our approach has been evaluated through the % two 
+% AUMP~\cite{Fillatre:2012:ASL:2333143.2333587}
+% and
+Ensemble Classifier~\cite{DBLP:journals/tifs/KodovskyFH12} based steganalyser.
+Its  particularization to spatial domain is 
+considered as state of the art steganalysers.
+\JFC{Features that are embedded into this steganalysis process 
+are CCPEV and SPAM features as described 
+in~\cite{DBLP:dblp_conf/mediaforensics/KodovskyPF10}.
+These one are extracted from the 
+set of cover images and the set of training images.}
+Next a small 
+set of weak classifiers is randomly built,
+each one working on a subspace of all the features.
+The final classifier is constructed by a majority voting 
+between the decisions of these  individual classifiers.
+
+
+%The former approach is based on a simplified parametric model of natural images.
+% Parameters are firstly estimated and an adaptive Asymptotically Uniformly Most Powerful
+% (AUMP) test is designed (theoretically and practically), to check whether
+% an image has stego content or not.
+% This approach is dedicated to verify whether LSB has been modified or not.
+% , the authors show that the 
+% machine learning step, which is often
+% implemented as a support vector machine,
+% can be favorably executed thanks to an ensemble classifier.
 
-%citer le second tableau, comparer avec EAISLSBMR
 
 \begin{table*}
 \begin{center}
-%\begin{small}
-\begin{tabular}{|c|c|c|c|c|c|c|c|}
+\begin{small}
+\setlength{\tabcolsep}{3pt}
+\begin{tabular}{|c|c|c|c|c|c|c|c|c|c|c|c|c|}
 \hline
-Schemes &amp; \multicolumn{3}{|c|}{STABYLO} &amp; \multicolumn{2}{|c|}{HUGO}&amp; \multicolumn{2}{|c|}{EAISLSBMR}\\
+Schemes &amp; \multicolumn{4}{|c|}{STABYLO} &amp; \multicolumn{2}{|c|}{HUGO}&amp; \multicolumn{2}{|c|}{EAISLSBMR} &amp;  \multicolumn{2}{|c|}{WOW} &amp;  \multicolumn{2}{|c|}{UNIWARD}\\
 \hline
-Embedding &amp;   \multicolumn{2}{|c|}{Adaptive} &amp; Fixed &amp; \multicolumn{2}{|c|}{Fixed}&amp; \multicolumn{2}{|c|}{Fixed} \\
+Embedding &amp; Fixed &amp;   \multicolumn{3}{|c|}{Adaptive (about 6.35\%)}  &amp; Fixed &amp; Adapt. &amp; Fixed &amp; Adapt. &amp; Fixed &amp; Adapt. &amp; Fixed &amp; Adapt. \\
 \hline
-Rate &amp;   + STC &amp;  + sample &amp; 10\% &amp; 10\%&amp; 6.35\%&amp; 10\%&amp; 6.35\%\\ 
+Rate &amp; 10\% &amp;  + sample &amp;   +STC(7) &amp; +STC(6)   &amp; 10\%&amp; $\approx$6.35\%&amp; 10\%&amp; $\approx$6.35\% &amp; 10\%&amp; $\approx$6.35\%&amp; 10\%&amp; $\approx$6.35\%\\ 
 \hline
-AUMP &amp; 0.39  &amp; 0.33  &amp; 0.22     &amp;  0.50 &amp;  0.50 &amp; 0.49 &amp; 0.50 \\
-\hline
-Ensemble Classifier &amp; \textbf{0.47} &amp; 0.44 &amp; 0.35     &amp; 0.48 &amp;  0.49  &amp;  0.43  &amp; 0.46 \\
+%AUMP &amp; 0.22 &amp; 0.33 &amp; 0.39  &amp;   0.45    &amp;  0.50 &amp;  0.50 &amp; 0.49 &amp; 0.50 \\
+%\hline
+Ensemble Classifier &amp; 0.35 &amp; 0.44 &amp; 0.47 &amp; 0.47     &amp; 0.48 &amp;  0.49  &amp;  0.43  &amp; 0.47 &amp; 0.48 &amp; 0.49 &amp; 0.46 &amp; 0.49 \\
 
 \hline
 \end{tabular}
-%\end{small}
+\end{small}
 \end{center}
 \caption{Steganalysing STABYLO\label{table:steganalyse}} 
 \end{table*}
 
 
-Results show that our approach is more easily detectable than HUGO, which
-is the most secure steganographic tool, as far as we know. However due to its 
-huge number of features integration, it is not lightweight, which justifies 
-in the authors' opinion the consideration of the proposed method.   
+Results of average testing errors 
+are summarized in Table~\ref{table:steganalyse}.
+First of all, STC outperforms the sample strategy %for % the two steganalysers
+ as 
+already noticed in the quality analysis presented in the previous section. 
+Next, our approach is more easily detectable than HUGO, 
+WOW and UNIWARD which are the most secure steganographic tool, 
+as far as we know. 
+However by combining Adaptive and STC strategies
+our approach obtains similar results than the ones of these schemes.
 
+Compared to EAILSBMR, we obtain similar
+results when the strategy is 
+Adaptive. 
+However due to its huge number of integration features, it is not lightweight.
+
+All these numerical experiments confirm 
+the objective presented in the motivations:
+providing an efficient steganography approach in a lightweight manner.
+
+\RC{In Figure~\ref{fig:error}, Ensemble Classifier has been used with all the previsou steganalizers with 3 different payloads. It can be observed that with important payload, STABYLO is not efficient, but as mentionned its complexity is far more simple compared to other tools.\\
+\begin{figure}
+\begin{center}
+\includegraphics[scale=0.5]{error}
+\end{center}
+\caption{Error obtained by Ensemble classifier with WOW/UNIWARD, HUGO, and STABYLO and different paylaods.}
+\label{fig:error} 
+\end{figure}
+}
</pre></body></html>