[canny.git] / experiments.tex

First of all, the whole code of STABYLO can be downloaded
\footnote{\url{http://http://members.femto-st.fr/jf-couchot/en/stabylo}}.
For all the experiments, the whole set of 10,000 images 
of the BOSS contest~\cite{Boss10} database is taken. 
In this set, each cover is a $512\times 512$
grayscale digital image in a RAW format. 
We restrict experiments to 
this set of cover images since this paper is more focused on 
the methodology than on benchmarks.    

We use the matrices $\hat{H}$ 
generated by the integers given
in Table~\ref{table:matrices:H}
as introduced in~\cite{FillerJF11}, since these ones have experimentally 
be proven to have the strongest modification efficiency.
For instance if the rate between the size of the message and the size of the 
cover vector
is 1/4, each number in $\{81, 95, 107, 121\}$ is translated into a binary number 
and each one constitutes thus a column of $\hat{H}$. 

\begin{table}
$$
\begin{array}{|l|l|}
\hline
\textrm{Rate} & \textrm{Matrix generators} \\
\hline
{1}/{2} & \{71,109\}\\
\hline
{1}/{3} & \{95, 101, 121\}\\
\hline
{1}/{4} & \{81, 95, 107, 121\}\\
\hline
{1}/{5} & \{75, 95, 97, 105, 117\}\\
\hline
{1}/{6} & \{73, 83, 95, 103, 109, 123\}\\
\hline
{1}/{7} & \{69, 77, 93, 107, 111, 115, 121\}\\
\hline
{1}/{8} & \{69, 79, 81, 89, 93, 99, 107, 119\}\\
\hline
{1}/{9} & \{69, 79, 81, 89, 93, 99, 107, 119, 125\}\\
\hline
\end{array}
$$
\caption{Matrix Generator for $\hat{H}$ in STC}\label{table:matrices:H}
\end{table}


Our approach is always compared to HUGO, to EAISLSBMR, to WOW and to UNIWARD
for the two strategies Fixed and Adaptive. 
For the former one, the payload has been set to 10\%. 
For the latter one, the Canny parameter $T$ has been set to 3. 
When $b$ is 7, the average size of the message that can be embedded
is 16,445 bits, 
that corresponds to an  average payload of 6.35\%.
For each cover image the STABYLO's embedding rate with these two parameters 
is memorized. 
Next each steganographic scheme is executed to produce the stego content of 
this cover with respect to this embedding rate.  



 

\subsection{Image quality}\label{sub:quality}
The visual quality of the STABYLO scheme is evaluated in this section.
For the sake of completeness, three metrics are computed in these experiments: 
the Peak Signal to Noise Ratio (PSNR), 
the PSNR-HVS-M family~\cite{psnrhvsm11}, 
and 
the weighted PSNR (wPSNR)~\cite{DBLP:conf/ih/PereiraVMMP01}.
The first one is widely used but does not take into
account the Human Visual System (HVS).
The other ones have been designed to tackle this problem.

If we apply them on the running example with the Adaptive and STC strategies, 
the PSNR, PSNR-HVS-M, and wPSNR values are respectively equal to 
68.39, 79.85, and 89.71 for the stego Lena when $b$ is equal to 7.
If $b$ is 6, these values are respectively equal to 
65.43, 77.2, and 89.35.




\begin{table*}
\begin{center}
\begin{small}
\setlength{\tabcolsep}{3pt}
\begin{tabular}{|c|c|c||c|c|c|c|c|c|c|c|c|c|}
\hline
Schemes & \multicolumn{4}{|c|}{STABYLO} & \multicolumn{2}{|c|}{HUGO}& \multicolumn{2}{|c|}{EAISLSBMR} & \multicolumn{2}{|c|}{WOW} & \multicolumn{2}{|c|}{UNIWARD}\\
\hline
Embedding &   Fixed & \multicolumn{3}{|c|}{Adaptive (about 6.35\%)} &  Fixed &Adaptive & Fixed &Adaptive & Fixed &Adaptive & Fixed &Adaptive \\
\hline
Rate &   10\% &  + sample &  +STC(7) & +STC(6) &  10\%&$\approx$6.35\%& 10\%&$\approx$6.35\%& 10\%&$\approx$6.35\%& 10\%&$\approx$6.35\%\\ 
\hline
PSNR & 61.86 & 63.48 &  66.55  &  63.7  & 64.65 & {67.08} & 60.8 & 62.9&65.9 & 68.3 & 65.8 & 69.2\\ 
\hline
PSNR-HVS-M & 72.9 & 75.39 & 78.6  & 75.5  & 76.67 & {79.6} & 71.8  & 76.0 &
76.7 & 80.35 & 77.6 & 81.2 \\
\hline
wPSNR & 77.47 & 80.59 & 86.43& 86.28  & 83.03 & {88.6} & 76.7 & 83& 83.8 & 90.4 & 85.2 & 91.9\\ 
\hline
\end{tabular}
\end{small}
\end{center}
\caption{Quality measures of steganography approaches\label{table:quality}}
\end{table*}



Results are summarized in Table~\ref{table:quality}.
In this table, STC(7) stands for embedding data in the LSB whereas
in STC(6), data are hidden in the last two  significant bits. 


Let us give an interpretation of these experiments.
First of all, the Adaptive strategy produces images with lower distortion 
than the images resulting from the 10\% fixed strategy.
Numerical results are indeed always greater for the former strategy than 
for the latter one.
These results are not surprising since the Adaptive strategy aims at 
embedding messages whose length is decided according to a higher threshold
into the edge detection.  


If we combine Adaptive and STC strategies 
the STABYLO scheme  provides images whose quality is higher than 
the EAISLSBMR's one but lower than the quality of high complexity 
schemes. Notice that the quality of the less respectful scheme (EAILSBMR) 
is lower than 6\% than the one of the most one.


% Let us now compare the STABYLO approach with other edge based steganography
% approaches, namely~\cite{DBLP:journals/eswa/ChenCL10,Chang20101286}.
% These two schemes focus on increasing the
% payload while the PSNR is acceptable, but do not 
% give quality metrics for fixed embedding rates from a large base of images. 




\subsection{Steganalysis}\label{sub:steg}



The steganalysis quality of our approach has been evaluated through the % two 
% AUMP~\cite{Fillatre:2012:ASL:2333143.2333587}
% and
Ensemble Classifier~\cite{DBLP:journals/tifs/KodovskyFH12} based steganalyser.
Its  particularization to spatial domain is 
considered as state of the art steganalysers.
Firstly, a space 
of 686 co-occurrence and Markov features is extracted from the 
set of cover images and the set of training images. Next a small 
set of weak classifiers is randomly built,
each one working on a subspace of all the features.
The final classifier is constructed by a majority voting 
between the decisions of these  individual classifiers.


%The former approach is based on a simplified parametric model of natural images.
% Parameters are firstly estimated and an adaptive Asymptotically Uniformly Most Powerful
% (AUMP) test is designed (theoretically and practically), to check whether
% an image has stego content or not.
% This approach is dedicated to verify whether LSB has been modified or not.
% , the authors show that the 
% machine learning step, which is often
% implemented as a support vector machine,
% can be favorably executed thanks to an ensemble classifier.


\begin{table*}
\begin{center}
\begin{small}
\setlength{\tabcolsep}{3pt}
\begin{tabular}{|c|c|c|c|c|c|c|c|c|c|c|c|c|}
\hline
Schemes & \multicolumn{4}{|c|}{STABYLO} & \multicolumn{2}{|c|}{HUGO}& \multicolumn{2}{|c|}{EAISLSBMR} &  \multicolumn{2}{|c|}{WOW} &  \multicolumn{2}{|c|}{UNIWARD}\\
\hline
Embedding & Fixed &   \multicolumn{3}{|c|}{Adaptive (about 6.35\%)}  & Fixed & Adapt. & Fixed & Adapt. & Fixed & Adapt. & Fixed & Adapt. \\
\hline
Rate & 10\% &  + sample &   +STC(7) & +STC(6)   & 10\%& $\approx$6.35\%& 10\%& $\approx$6.35\% & 10\%& $\approx$6.35\%& 10\%& $\approx$6.35\%\\ 
\hline
%AUMP & 0.22 & 0.33 & 0.39  &   0.45    &  0.50 &  0.50 & 0.49 & 0.50 \\
%\hline
Ensemble Classifier & 0.35 & 0.44 & 0.47 & 0.47     & 0.48 &  0.49  &  0.43  & 0.47 & 0.48 & 0.49 & 0.46 & 0.49 \\

\hline
\end{tabular}
\end{small}
\end{center}
\caption{Steganalysing STABYLO\label{table:steganalyse}} 
\end{table*}


Results of average testing errors 
are summarized in Table~\ref{table:steganalyse}.
First of all, STC outperforms the sample strategy %for % the two steganalysers
 as 
already noticed in the quality analysis presented in the previous section. 
Next, our approach is more easily detectable than HUGO, 
WOW and UNIWARD which are the most secure steganographic tool, 
as far as we know. 
However by combining Adaptive and STC strategies
our approach obtains similar results than the ones of these schemes.

Compared to EAILSBMR, we obtain similar
results when the strategy is 
Adaptive. 
However due to its huge number of integration features, it is not lightweight.

All these numerical experiments confirm 
the objective presented in the motivations:
providing an efficient steganography approach in a lightweight manner.