X-Git-Url: https://bilbo.iut-bm.univ-fcomte.fr/and/gitweb/canny.git/blobdiff_plain/f4086cd758c530e5056b03d442f1cb5cf07b447b..6bc78c180bea99d493483c3cac3f559301d3af09:/experiments.tex?ds=sidebyside diff --git a/experiments.tex b/experiments.tex index 0c08d69..aa49e84 100644 --- a/experiments.tex +++ b/experiments.tex @@ -1,184 +1,236 @@ -For whole experiments, the whole set of 10000 images +First of all, the whole code of STABYLO can be downloaded +\footnote{\url{http://http://members.femto-st.fr/jf-couchot/en/stabylo}}. +For all the experiments, the whole set of 10,000 images of the BOSS contest~\cite{Boss10} database is taken. In this set, each cover is a $512\times 512$ grayscale digital image in a RAW format. We restrict experiments to this set of cover images since this paper is more focused on -the methodology than benchmarking. -Our approach is always compared to Hugo~\cite{DBLP:conf/ih/PevnyFB10} -and to EAISLSBMR~\cite{Luo:2010:EAI:1824719.1824720}. - - - - - -\subsection{Adaptive Embedding Rate} -Two strategies have been developed in our scheme, -depending on the embedding rate that is either \emph{adaptive} or \emph{fixed}. - -In the former the embedding rate depends on the number of edge pixels. -The higher it is, the larger the message length that can be inserted is. -Practically, a set of edge pixels is computed according to the -Canny algorithm with an high threshold. -The message length is thus defined to be half of this set cardinality. -In this strategy, two methods are thus applied to extract bits that -are modified. The first one is a direct application of the STC algorithm. -This method is further referred to as \emph{adaptive+STC}. -The second one randomly chooses the subset of pixels to modify by -applying the BBS PRNG again. This method is denoted \emph{adaptive+sample}. -Notice that the rate between -available bits and bit message length is always equal to 2. -This constraint is indeed induced by the fact that the efficiency -of the STC algorithm is unsatisfactory under that threshold. -In our experiments and with the adaptive scheme, -the average size of the message that can be embedded is 16,445 bits. -Its corresponds to an average payload of 6.35\%. - - - - -In the latter, the embedding rate is defined as a percentage between the -number of modified pixels and the length of the bit message. -This is the classical approach adopted in steganography. -Practically, the Canny algorithm generates -a set of edge pixels related to a threshold that is decreasing until its cardinality -is sufficient. If the set cardinality is more than twice larger than the -bit message length, a STC step is again applied. -Otherwise, pixels are again randomly chosen with BBS. - - - -\subsection{Image Quality} -The visual quality of the STABYLO scheme is evaluated in this section. -For the sake of completeness, four metrics are computed in these experiments: -the Peak Signal to Noise Ratio (PSNR), -the PSNR-HVS-M family~\cite{psnrhvsm11}, -%the BIQI~\cite{MB10}, -and -the weighted PSNR (wPSNR)~\cite{DBLP:conf/ih/PereiraVMMP01}. -The first one is widely used but does not take into -account the Human Visual System (HVS). -The other ones have been designed to tackle this problem. - - - - -\begin{table*} -\begin{center} -\begin{tabular}{|c|c|c||c|c|c|c|c|} +the methodology than on benchmarks. + +We use the matrices $\hat{H}$ +generated by the integers given +in Table~\ref{table:matrices:H} +as introduced in~\cite{FillerJF11}, since these ones have experimentally +be proven to have the strongest modification efficiency. +For instance if the rate between the size of the message and the size of the +cover vector +is 1/4, each number in $\{81, 95, 107, 121\}$ is translated into a binary number +and each one constitutes thus a column of $\hat{H}$. + +\begin{table} +$$ +\begin{array}{|l|l|} \hline -Schemes & \multicolumn{3}{|c|}{STABYLO} & \multicolumn{2}{|c|}{HUGO}& \multicolumn{2}{|c|}{EAISLSBMR} \\ +\textrm{Rate} & \textrm{Matrix generators} \\ \hline -Embedding & Fixed & \multicolumn{2}{|c|}{Adaptive} & \multicolumn{2}{|c|}{Fixed}& \multicolumn{2}{|c|}{Fixed} \\ +{1}/{2} & \{71,109\}\\ \hline -Rate & 10\% & + sample & + STC & 10\%&6.35\%& 10\%&6.35\%\\ +{1}/{3} & \{95, 101, 121\}\\ \hline -PSNR & 61.86 & 63.48 & 66.55 (\textbf{-0.8\%}) & 64.65 & {67.08} & 60.8 & 62.9\\ +{1}/{4} & \{81, 95, 107, 121\}\\ \hline -PSNR-HVS-M & 72.9 & 75.39 & 78.6 (\textbf{-0.8\%}) & 76.67 & {79.23} & 61.3 & 63.4\\ -%\hline -%BIQI & 28.3 & 28.28 & 28.4 & 28.28 & 28.28 & 28.2 & 28.2\\ +{1}/{5} & \{75, 95, 97, 105, 117\}\\ \hline -wPSNR & 77.47 & 80.59 & 86.43(\textbf{-1.6\%}) & 83.03 & {87.8} & 76.7 & 80.6\\ +{1}/{6} & \{73, 83, 95, 103, 109, 123\}\\ \hline -\end{tabular} - -\begin{footnotesize} -\vspace{2em} -Variances given in bold font express the quality differences between -HUGO and STABYLO with STC+adaptive parameters. -\end{footnotesize} - -\end{center} -\caption{Quality Measures of Steganography Approaches\label{table:quality}} -\end{table*} - - - -Results are summarized into the Table~\ref{table:quality}. -Let us give an interpretation of these first experiments. -First of all, the adaptive strategy produces images with lower distortion -than the one of images resulting from the 10\% fixed strategy. -Numerical results are indeed always greater for the former strategy than -for the latter. -These results are not surprising since the adaptive strategy aims at -embedding messages whose length is decided according to an higher threshold -into the edge detection. -Let us focus on the quality of HUGO images: with a given fixed -embedding rate (10\%), -HUGO always produces images whose quality is higher than the STABYLO's one. -However our approach always outperforms EAISLSBMR since this one may modify -the two least significant bits whereas STABYLO only alter LSB. - -If we combine \emph{adaptive} and \emph{STC} strategies -(which leads to an average embedding rate equal to 6.35\%) -our approach provides equivalent metrics than HUGO. -The quality variance between HUGO and STABYLO for these parameters -is given in bold font. It is always close to 1\% which confirms -the objective presented in the motivations: -providing an efficient steganography approach with a lightweight manner. - - -Let us now compare the STABYLO approach with other edge based steganography -approaches, namely~\cite{DBLP:journals/eswa/ChenCL10,Chang20101286}. -These two schemes focus on increasing the -payload while the PSNR is acceptable, but do not -give quality metrics for fixed embedding rates from a large base of images. - +{1}/{7} & \{69, 77, 93, 107, 111, 115, 121\}\\ +\hline +{1}/{8} & \{69, 79, 81, 89, 93, 99, 107, 119\}\\ +\hline +{1}/{9} & \{69, 79, 81, 89, 93, 99, 107, 119, 125\}\\ +\hline +\end{array} +$$ +\caption{Matrix Generator for $\hat{H}$ in STC.}\label{table:matrices:H} +\end{table} +Our approach is always compared to HUGO, to EAISLSBMR, to WOW and to UNIWARD +for the two strategies Fixed and Adaptive. +For the former one, the payload has been set to 10\%. +For the latter one, the Canny parameter $T$ has been set to 3. +When $b$ is 7, the average size of the message that can be embedded +is 16,445 bits, +that corresponds to an average payload of 6.35\%. +For each cover image the STABYLO's embedding rate with these two parameters +is memorized. +Next each steganographic scheme is executed to produce the stego content of +this cover with respect to this embedding rate. -\subsection{Steganalysis} + -The steganalysis quality of our approach has been evaluated through the two -AUMP~\cite{Fillatre:2012:ASL:2333143.2333587} -and Ensemble Classifier~\cite{DBLP:journals/tifs/KodovskyFH12} based steganalysers. -Both aims at detecting hidden bits in grayscale natural images and are -considered as the state of the art of steganalysers in spatial domain~\cite{FK12}. -The former approach is based on a simplified parametric model of natural images. -Parameters are firstly estimated and an adaptive Asymptotically Uniformly Most Powerful -(AUMP) test is designed (theoretically and practically), to check whether -an image has stego content or not. -This approach is dedicated to verify whether LSB has been modified or not. -In the latter, the authors show that the -machine learning step, which is often -implemented as support vector machine, -can be favorably executed thanks to an ensemble classifier. +% \subsection{Image quality}\label{sub:quality} +% The visual quality of the STABYLO scheme is evaluated in this section. +% For the sake of completeness, three metrics are computed in these experiments: +% the Peak Signal to Noise Ratio (PSNR), +% the PSNR-HVS-M family~\cite{psnrhvsm11}, +% and +% the weighted PSNR (wPSNR)~\cite{DBLP:conf/ih/PereiraVMMP01}. +% The first one is widely used but does not take into +% account the Human Visual System (HVS). +% The other ones have been designed to tackle this problem. + +% If we apply them on the running example with the Adaptive and STC strategies, +% the PSNR, PSNR-HVS-M, and wPSNR values are respectively equal to +% 68.39, 79.85, and 89.71 for the stego Lena when $b$ is equal to 7. +% If $b$ is 6, these values are respectively equal to +% 65.43, 77.2, and 89.35. + + + + +% \begin{table*} +% \begin{center} +% \begin{small} +% \setlength{\tabcolsep}{3pt} +% \begin{tabular}{|c|c|c||c|c|c|c|c|c|c|c|c|c|} +% \hline +% Schemes & \multicolumn{4}{|c|}{STABYLO} & \multicolumn{2}{|c|}{HUGO}& \multicolumn{2}{|c|}{EAISLSBMR} & \multicolumn{2}{|c|}{WOW} & \multicolumn{2}{|c|}{UNIWARD}\\ +% \hline +% Embedding & Fixed & \multicolumn{3}{|c|}{Adaptive (about 6.35\%)} & Fixed &Adaptive & Fixed &Adaptive & Fixed &Adaptive & Fixed &Adaptive \\ +% \hline +% Rate & 10\% & + sample & +STC(7) & +STC(6) & 10\%&$\approx$6.35\%& 10\%&$\approx$6.35\%& 10\%&$\approx$6.35\%& 10\%&$\approx$6.35\%\\ +% \hline +% PSNR & 61.86 & 63.48 & 66.55 & 63.7 & 64.65 & {67.08} & 60.8 & 62.9&65.9 & 68.3 & 65.8 & 69.2\\ +% \hline +% PSNR-HVS-M & 72.9 & 75.39 & 78.6 & 75.5 & 76.67 & {79.6} & 71.8 & 76.0 & +% 76.7 & 80.35 & 77.6 & 81.2 \\ +% \hline +% wPSNR & 77.47 & 80.59 & 86.43& 86.28 & 83.03 & {88.6} & 76.7 & 83& 83.8 & 90.4 & 85.2 & 91.9\\ +% \hline +% \end{tabular} +% \end{small} +% \end{center} +% \caption{Quality measures of steganography approaches\label{table:quality}} +% \end{table*} + + + +% Results are summarized in Table~\ref{table:quality}. +% In this table, STC(7) stands for embedding data in the LSB whereas +% in STC(6), data are hidden in the last two significant bits. + + +% Let us give an interpretation of these experiments. +% First of all, the Adaptive strategy produces images with lower distortion +% than the images resulting from the 10\% fixed strategy. +% Numerical results are indeed always greater for the former strategy than +% for the latter one. +% These results are not surprising since the Adaptive strategy aims at +% embedding messages whose length is decided according to a higher threshold +% into the edge detection. + + +% If we combine Adaptive and STC strategies +% the STABYLO scheme provides images whose quality is higher than +% the EAISLSBMR's one but lower than the quality of high complexity +% schemes. Notice that the quality of the less respectful scheme (EAILSBMR) +% is lower than 6\% than the one of the most one. + + +% % Let us now compare the STABYLO approach with other edge based steganography +% % approaches, namely~\cite{DBLP:journals/eswa/ChenCL10,Chang20101286}. +% % These two schemes focus on increasing the +% % payload while the PSNR is acceptable, but do not +% % give quality metrics for fixed embedding rates from a large base of images. + + + + +\subsection{Steganalysis}\label{sub:steg} + + + +The steganalysis quality of our approach has been evaluated through the % two +% AUMP~\cite{Fillatre:2012:ASL:2333143.2333587} +% and +Ensemble Classifier~\cite{DBLP:journals/tifs/KodovskyFH12} based steganalyser. +Its particularization to spatial domain is +considered as state of the art steganalysers. +Features that are embedded into this steganalysis process +are CCPEV and SPAM features as described +in~\cite{DBLP:dblp_conf/mediaforensics/KodovskyPF10}. +They are extracted from the +set of cover images and the set of training images. +Next a small +set of weak classifiers is randomly built, +each one working on a subspace of all the features. +The final classifier is constructed by a majority voting +between the decisions of these individual classifiers. + + +%The former approach is based on a simplified parametric model of natural images. +% Parameters are firstly estimated and an adaptive Asymptotically Uniformly Most Powerful +% (AUMP) test is designed (theoretically and practically), to check whether +% an image has stego content or not. +% This approach is dedicated to verify whether LSB has been modified or not. +% , the authors show that the +% machine learning step, which is often +% implemented as a support vector machine, +% can be favorably executed thanks to an ensemble classifier. \begin{table*} \begin{center} -%\begin{small} -\begin{tabular}{|c|c|c|c|c|c|c|c|} -\hline -Schemes & \multicolumn{3}{|c|}{STABYLO} & \multicolumn{2}{|c|}{HUGO}& \multicolumn{2}{|c|}{EAISLSBMR}\\ +\begin{small} +\setlength{\tabcolsep}{3pt} +\begin{tabular}{|c|c|c|c|c|c|c|c|c|c|c|c|c|} \hline -Embedding & Fixed & \multicolumn{2}{|c|}{Adaptive} & \multicolumn{2}{|c|}{Fixed}& \multicolumn{2}{|c|}{Fixed} \\ +Schemes & \multicolumn{4}{|c|}{STABYLO} & \multicolumn{2}{|c|}{HUGO}& \multicolumn{2}{|c|}{EAISLSBMR} & \multicolumn{2}{|c|}{WOW} & \multicolumn{2}{|c|}{UNIWARD}\\ \hline -Rate & 10\% & + sample & + STC & 10\%& 6.35\%& 10\%& 6.35\%\\ +Embedding & Fixed & \multicolumn{3}{|c|}{Adaptive (about 6.35\%)} & Fixed & Adapt. & Fixed & Adapt. & Fixed & Adapt. & Fixed & Adapt. \\ \hline -AUMP & 0.22 & 0.33 & 0.39 & 0.50 & 0.50 & 0.49 & 0.50 \\ +Rate & 10\% & + sample & +STC(7) & +STC(6) & 10\%& $\approx$6.35\%& 10\%& $\approx$6.35\% & 10\%& $\approx$6.35\%& 10\%& $\approx$6.35\%\\ \hline -Ensemble Classifier & 0.35 & 0.44 & 0.47 & 0.48 & 0.49 & 0.43 & 0.46 \\ +%AUMP & 0.22 & 0.33 & 0.39 & 0.45 & 0.50 & 0.50 & 0.49 & 0.50 \\ +%\hline +Ensemble Classifier & 0.35 & 0.44 & 0.47 & 0.47 & 0.48 & 0.49 & 0.43 & 0.47 & 0.48 & 0.49 & 0.46 & 0.49 \\ \hline \end{tabular} -%\end{small} +\end{small} \end{center} -\caption{Steganalysing STABYLO\label{table:steganalyse}} +\caption{Steganalysing STABYLO\label{table:steganalyse}.} \end{table*} -Results are summarized in Table~\ref{table:steganalyse}. -First of all, STC outperforms the sample strategy for the two steganalysers, as +Results of average testing errors +are summarized in Table~\ref{table:steganalyse}. +First of all, STC outperforms the sample strategy %for % the two steganalysers + as already noticed in the quality analysis presented in the previous section. -Next, our approach is more easily detectable than HUGO, which -is the most secure steganographic tool, as far as we know. -However by combining \emph{adaptive} and \emph{STC} strategies -our approach obtains similar results than HUGO ones. -However due to its -huge number of features integration, it is not lightweight, which justifies -in the authors' opinion the consideration of the proposed method. - +Next, our approach is more easily detectable than HUGO, +WOW and UNIWARD which are the most secure steganographic tool, +as far as we know. +However by combining Adaptive and STC strategies +our approach obtains similar results than the ones of these schemes. + +Compared to EAILSBMR, we obtain similar +results when the strategy is +Adaptive. +However due to its huge number of integration features, it is not lightweight. + +All these numerical experiments confirm +the objective presented in the motivations: +providing an efficient steganography approach in a lightweight manner +for small payload. + +In Figure~\ref{fig:error}, +Ensemble Classifier has been used with all the previous +steganographic schemes with 4 different payloads. +It can be observed that face to high values of payload, +STABYLO is definitely not secure enough. +However thanks to an efficient very low-complexity (Fig.\ref{fig:compared}), +we argue that the user should embed tiny messages in many images +than a larger message in only one image. +\begin{figure} +\begin{center} +\includegraphics[scale=0.5]{error} +\end{center} +\caption{Testing errors obtained by Ensemble classifier with +WOW/UNIWARD, HUGO, and STABYLO w.r.t. payload.} +\label{fig:error} +\end{figure}