This section is devoted to the robustness study of our scheme.
This one has to ensure that the watermark withstands against 
different types of active attacks that modify the watermarked image.

For the whole experiment, a set of 100 images is randomly extracted 
from the database taken from the BOSS contest~\cite{Boss10}. 
In this set, each cover is a $512\times 512$
grayscale digital image.
The considered watermark $m$ is given in Fig.~\ref{(b) Watermark}. 
Testing the robustness of the approach is achieved by successively applying
on watermarked images attacks like cropping, compression, geometric 
transformations,\ldots % (from Sec.~\ref{sub:crop} to Sec.~\ref{sub:lsb}).




\begin{figure}
\begin{center}
\includegraphics[width=0.06\textwidth]{exp/invader}
\end{center}
\caption{The Watermark}
\label{(b) Watermark}
\end{figure}


Differences between $m$ and $m'$ are 
computed. Behind a given threshold rate, the image is said to be watermarked.  
%Finally, 
Discussion on metric quality of the approach is finally given.
%Sect.~\ref{sub:roc}.

\begin{remark}
In the remainder of this article, on following figures, the difference percentage
corresponds to the distance between the retrieved and the original watermarks.
\end{remark}
 
%\subsection{Robustness against Cropping}\label{sub:crop}
%\subsection{Against Cropping Attack}\label{sub:crop}

Robustness of the approach is  evaluated by
applying different percentages of cropping: from 0.25\% to 90\%.
Results are given in Fig.~\ref{Fig:atck:dec}, %.
%, and Fig.~\ref{Fig:atq:dec:img} gives the cropped image 
%where 36\% of the image is removed.
%Fig.~\ref{Fig:atq:dec:curves}
which presents effects of such an attack.
All the percentage differences are so far less than 97\% 
and thus robustness is established.



\begin{figure}[ht]
  \centering
  % \subfigure[Cropped Image.]{\includegraphics[width=0.24\textwidth]
  %   {5007_dec_307.png}\label{Fig:atq:dec:img}}\hspace{2cm}
%  \subfigure[Cropped Image.]{\includegraphics[width=0.24\textwidth]
  %  {5007_dec_307}\label{Fig:atq:dec:img}}\hspace{2cm}
%  \subfigure[Cropping Effect]{
\includegraphics[width=0.35\textwidth]{exp/atq-dec}\label{Fig:atq:dec:curves}%}
%\includegraphics[width=0.5\textwidth]{exp/atq-dec.eps}\label{Fig:atq:dec:curves}}
\caption{Cropping Results}
\label{Fig:atck:dec}
\end{figure}


%\subsection{Robustness against compression}\label{sub:comp}
%\subsection{Against Compression Attack}\label{sub:comp}

Robustness against compression is addressed
by studying both JPEG  and JPEG 2000 image compression.
Results are respectively presented in Fig.~\ref{Fig:atq:jpg:curves}
and Fig.~\ref{Fig:atq:jp2:curves}.
It is not hard to see that robustness is well established for 
JPEG2000 compression: for all the ratio larger than 10\%, the watermark 
is retrieved.  
However, our scheme is not robust against JPEG compression for a ratio inferior
to 90\%.

A potential solution in order to improve this result should be to insert the
watermark in least significant coefficient of the image described in frequency
domain as for example with discrete cosine transform or with wavelet transform.
This study will be described in future works.
%\JFC{voir comment amméliorer}
\begin{figure}[ht]
  \centering
  \subfigure[JPEG Effect]{
\includegraphics[width=0.35\textwidth]{exp/atq-jpg}\label{Fig:atq:jpg:curves}}
%\includegraphics[width=0.45\textwidth]{exp/atq-jpg.eps}\label{Fig:atq:jpg:curves}}
  \subfigure[JPEG 2000 Effect]{
\includegraphics[width=0.35\textwidth]{exp/atq-jp2}\label{Fig:atq:jp2:curves}}
%\includegraphics[width=0.45\textwidth]{exp/atq-jp2.eps}\label{Fig:atq:jp2:curves}}
\caption{Compression Results}
\label{Fig:atck:comp}
\end{figure}



%\subsection{Robustness against Contrast and Sharpness
%Attack}\label{sub:contrast}
%\subsection{Against Contrast and Sharpness Attack}\label{sub:contrast}

% Contrast and Sharpness adjustment belong to the classical set of 
% filtering image attacks.
% Results of such attacks are presented in 
% Fig.~\ref{Fig:atq:fil}, where 
% Fig.~\ref{Fig:atq:cont:curve} and Fig.~\ref{Fig:atq:sh:curve} summarize 
% effects of contrast and sharpness adjustment respectively. 
% Robustness is not established for both these types of attacks.
% Future works give issues to tackle this problem.

% \begin{figure}[ht]
%   \centering
%   \subfigure[Contrast Effect]{
% \includegraphics[width=0.4\textwidth]{exp/atq-contrast}\label{Fig:atq:cont:curve}}
% %\includegraphics[width=0.45\textwidth]{exp/atq-contrast.eps}\label{Fig:atq:cont:curve}}
%   \subfigure[Sharpness Effect]{
% \includegraphics[width=0.4\textwidth]{exp/atq-flou}\label{Fig:atq:sh:curve}}
% %\includegraphics[width=0.45\textwidth]{exp/atq-flou.eps}\label{Fig:atq:sh:curve}}
% \caption{Filtering Results}
% \label{Fig:atq:fil}
% \end{figure}


%nom;../../../../BCG10/Oxford11/experiments/images/1247 donne des résultats 
%étonnants

%\subsection{Against Geometric Transformation}\label{sub:rotation}
Among geometric transformations, we focus on  
rotations, \textit{i.e.}, when two opposite rotations 
of angle $\theta$ are successively applied around the center of the image.
In these geometric transformations,  angles range from 2 to 60 
degrees.  
Results are presented in Fig.~\ref{Fig:atq:rot}. %: Fig.~\ref{Fig:atq:rot:img}
% gives the image of a rotation of 20 degrees whereas
% Fig.~\ref{Fig:atq:rot:curve} presents the effects of such an attack.
Thanks to an efficient embedding, our scheme is resistant to all
that type of attacks.



\begin{figure}[ht]
  \centering
%   \subfigure[20 degrees Rotation Image]{
%     \includegraphics[width=0.2\textwidth]{5007_rot_20}\label{Fig:atq:rot:img}}
% %    \includegraphics[width=0.25\textwidth]{5007_rot_10.eps}\label{Fig:atq:rot:img}}
% 
%   \subfigure[Rotation Effect]{
\includegraphics[width=0.35\textwidth]{exp/atq-rot}\label{Fig:atq:rot:curve}%}
%\includegraphics[width=0.45\textwidth]{exp/atq-rot.eps}\label{Fig:atq:rot:curve}}

\caption{Rotation Attack Results}
\label{Fig:atq:rot}
\end{figure}

%\subsection{Against LSB Transformation}\label{sub:lsb}
The first step of this scheme has defined $x$ as the LSB of the host and 
is thus based on LSB modifications.
This part focuses on two types of attacks modifying these LSB sets (see
Fig~\ref{Fig:atq:lsb}). The former consists in setting to zero a subset of this one. 
Results are expressed in Fig.~\ref{Fig:atq:lsb:curve} and 
show that the scheme is robust, unless 95\% of the LSB is erased.
In this case the image is really damaged.
The latter consists in applying again this scheme on the watermarked image 
but  with another message. 
Results of Fig.~\ref{Fig:atq:ci2:curve} show that this scheme is robust 
against that type of attack, provided the number of iterations is 
lesser than $1.75$ times the number of pixels.
With more iterations, the image is dramatically modified: more than 50\% 
of the LSB is switched. However, future works present ideas to tackle
this problem.


\begin{figure}[ht]
  \centering
  \subfigure[LSB Erasing Effect]{
    \includegraphics[width=0.35\textwidth]{exp/atq-lsb}\label{Fig:atq:lsb:curve}}

  \subfigure[Applying Algorithm twice]{
\includegraphics[width=0.35\textwidth]{exp/atq-ci2}\label{Fig:atq:ci2:curve}}
%\includegraphics[width=0.45\textwidth]{exp/atq-rot.eps}\label{Fig:atq:rot:curve}}

\caption{LSB Modifications}
\label{Fig:atq:lsb}
\end{figure}




\subsection{Evaluation of the  Embeddings}\label{sub:roc}
A Receiver Operating Characteristic approach has been implemented to find
the most adapted threshold w.r.t. the separation between  
watermarked images and other ones.

\begin{figure}[ht]
\begin{center}
\includegraphics[width=0.35\textwidth]{exp/ROC}
%\includegraphics[width=7cm]{ROC.eps}
\end{center}
\caption{ROC Curves for DWT or DCT Embeddings}\label{fig:roc:dwt}
\end{figure}

The Figure~\ref{fig:roc:dwt} is the Receiver Operating Characteristic (ROC) 
curve. This curve is close to the ideal one that is without 
False Positive and False Negative answer.
The threshold with best results is  a distance equal to 0,97. 
With such a value, we can give some confidence intervals
for most of 
 evaluated attacks. The
approach is resistant to all the cropping where percentage is less than 90,
to  JPEG200 compression where quality ratio is greater than 5\%,
to all the rotation attacks,
to LSB erasing when less than 95\% are set to 0,
a second application of the scheme with less than 1.75 iterations 
per pixel.

