\title{A new approach based on a least square method for real-time estimation of cantilever array deflections with a FPGA}
\author{\IEEEauthorblockN{Raphaël Couturier\IEEEauthorrefmark{1}, Stéphane
-Domas\IEEEauthorrefmark{1}, Gwenhaël Goavec-Merou\IEEEauthorrefmark{2} and
-Michel Lenczner\IEEEauthorrefmark{2}}
+Domas\IEEEauthorrefmark{1}, Gwenhaël Goavec-Merou\IEEEauthorrefmark{2}, Mélanie Favre\IEEEauthorrefmark{3},
+Michel Lenczner\IEEEauthorrefmark{2} and André Meister\IEEEauthorrefmark{3}} \\
\IEEEauthorblockA{\IEEEauthorrefmark{1}FEMTO-ST, DISC, University of Franche-Comte, Belfort, France \and
-\{raphael.couturier,stephane.domas\}@univ-fcomte.fr}
+\{raphael.couturier,stephane.domas\}@univ-fcomte.fr} \\
\IEEEauthorblockA{\IEEEauthorrefmark{2}FEMTO-ST, Time-Frequency, University of Franche-Comte, Besançon, France \and
-\{michel.lenczner@utbm.fr,gwenhael.goavec@trabucayre.com} }
+michel.lenczner@utbm.fr,gwenhael.goavec@trabucayre.com}\\
+\IEEEauthorblockA{\IEEEauthorrefmark{3}CSEM, Centre Suisse d’Electronique et de Microtechnique, Neuchatel, Switzerland \and
+\{melanie.favre,andre.meister\}@csem.ch}
+ }
\begin{abstract}
Atomic force microscopes (AFM) provide high resolution images of surfaces.
\end{IEEEkeywords}
\IEEEpeerreviewmaketitle
+%\maketitle
\section{Introduction}
based on spline has been introduced for cantilever position
estimation. The overall process gives accurate results but
computations are performed on a standard computer using LabView
-\textsuperscript{\textregistered} . Consequently, the main drawback
+\textsuperscript{\textregistered}. Consequently, the main drawback
of this implementation is that the computer is a bottleneck. In this
paper we pose the problem of real-time cantilever position estimation
and bring a hardware/software solution. It includes a fast method
process. Our solution based on the least square method and its
implementation on a FPGA is presented in
Section~\ref{sec:solus}. Numerical experimentations are described in
-Section~\ref{sec:results}. Finally a conclusion and some perspectives
+Section~\ref{sec:xp-test}. Finally a conclusion and some perspectives
are drawn.
\section{Architecture and goals}
\label{sec:measure}
In order to build simple, cost effective and user-friendly cantilever
-arrays, authors of ~\cite{AFMCSEM11} have developed a system based on
-interferometry.
+arrays, we use a system based on interferometry. The two following
+sections summarize the original characteristics of its architecture
+and computation method.
\subsection{Experimental setup}
induced by the vertical displacement of the cantilever.
The system is based on a Linnick interferometer~\cite{Sinclair:05}.
-It is illustrated in Figure~\ref{fig:AFM} \footnote{by courtesy of
- CSEM}. A laser diode is first split (by the splitter) into a
-reference beam and a sample beam both reaching the cantilever array.
-The complete system including a cantilever array\ and the optical
-system can be moved thanks to a translation and rotational hexapod
-stage with five degrees of freedom. Thus, the cantilever array is
-centered in the optical system which can be adjusted accurately. The
-beam illuminates the array by a microscope objective and the light
-reflects on the cantilevers. Likewise the reference beam reflects on a
-movable mirror. A CMOS camera chip records the reference and sample
-beams which are recombined in the beam splitter and the
-interferogram. At the beginning of each experiment, the movable mirror
-is fitted manually in order to align the interferometric fringes
-approximately parallel to the cantilevers. Then, cantilever motion in
-the transverse direction produces movements in the fringes. They are
-detected with the CMOS camera which images are analyzed by a Labview
-program to recover the cantilever deflections.
+It is illustrated in Figure~\ref{fig:AFM}. A laser diode is first
+split (by the splitter) into a reference beam and a sample beam both
+reaching the cantilever array. The complete system including a
+cantilever array and the optical system can be moved thanks to a
+translation and rotational hexapod stage with five degrees of
+freedom. Thus, the cantilever array is centered in the optical system
+which can be adjusted accurately. The beam illuminates the array by a
+microscope objective and the light reflects on the
+cantilevers. Likewise the reference beam reflects on a movable
+mirror. A CMOS camera chip records the reference and sample beams
+which are recombined in the beam splitter and the interferogram. At
+the beginning of each experiment, the movable mirror is fitted
+manually in order to align the interferometric fringes approximately
+parallel to the cantilevers. Then, cantilever motion in the transverse
+direction produces movements in the fringes. They are detected with
+the CMOS camera which images are analyzed by a Labview program to
+recover the cantilever deflections.
\begin{figure}[tbp]
\begin{center}
\label{fig:img-xp}
\end{figure}
-As shown in Figure \ref{fig:img-xp} \footnote{by courtesy of CSEM}, each
-cantilever is covered by several interferometric fringes. The fringes
-distort when cantilevers are deflected. In \cite{AFMCSEM11}, a novel
-method for interferometric based cantilever deflection measurement was
-reported. For each cantilever, the method uses three segments of pixels,
-parallel to its section, to determine phase shifts. The first is
-located just above the AFM tip (tip profile), it provides the phase
+As shown in Figure \ref{fig:img-xp}, each cantilever is covered by
+several interferometric fringes. The fringes distort when cantilevers
+are deflected. For each cantilever, the method uses three segments of
+pixels, parallel to its section, to determine phase shifts. The first
+is located just above the AFM tip (tip profile), it provides the phase
shift modulo $2\pi $. The second one is close to the base junction
(base profile) and is used to determine the exact multiple of $2\pi $
through an operation called unwrapping where it is assumed that the
is done after images have been taken at regular time steps. For each
image, the phase $\theta $ of all profiles is computed to obtain,
after unwrapping, the cantilever deflection. The phase determination
-in \cite{AFMCSEM11} is achieved by a spline based algorithm which is
-the most consuming part of the computation. In this article, we
-propose an alternate version based on the least square method which is
-faster and better suited for FPGA implementation. Moreover, it can be
-used in real-time, i.e. after each image picked by the camera.
+is achieved by a spline based algorithm, which is the most consuming
+part of the computation. In this article, we propose an alternate
+version based on the least square method which is faster and better
+suited for FPGA implementation. Moreover, it can be used in real-time,
+i.e. after each image is picked by the camera.
\subsection{Computation design goals}
\label{sec:goals}
To evaluate the solution performances, we choose a goal which consists
-in designing a computing unit able to estimate the deflections of
-a $10\times 10$%
--cantilever array, faster than the camera image stream. In addition,
-the result accuracy must be close to 0.3nm, the maximum precision
-reached in \cite{AFMCSEM11}. Finally, the latency between the entrance
-of the first pixel of an image and the end of deflection computation
-must be as small as possible. All these requirement are
-stated in the perspective of implementing real-time active control for
-each cantilever, see~\cite{LencznerChap10,Hui11}.
+in designing a computing unit able to estimate the deflections of a
+$10\times 10$-cantilever array, faster than the camera image
+stream. In addition, the result accuracy must be close to 0.3nm, the
+maximum precision reached in~\cite{AFMCSEM11}. Finally, the latency
+between the entrance of the first pixel of an image and the end of
+deflection computation must be as small as possible. All these
+requirement are stated in the perspective of implementing real-time
+active control for each cantilever, see~\cite{LencznerChap10,Hui11}.
If we put aside other hardware issues like the speed of the link
between the camera and the computation unit, the time to deserialize
pixels and to store them in memory, the phase computation is the
bottleneck of the whole process. For example, the camera in the setup
-of \cite{AFMCSEM11} provides $%
-1024\times 1204$ pixels with an exposition time of 2.5ms. Thus, if the
-pixel extraction time is neglected, each phase calculation of a
-100-cantilever array should take no more than 12.5$\mu$s.
+of \cite{AFMCSEM11} provides $1024\times 1204$ pixels with an
+exposition time of 2.5ms. Thus, if the pixel extraction time is
+neglected, each phase calculation of a 100-cantilever array should
+take no more than 12.5$\mu$s.
In fact, this timing is a very hard constraint. To illustrate this point, we
consider a very small program that initializes twenty million of doubles in
In order to evaluate the original algorithm, we translated it in C
language. As stated in section \ref{sec:algo-comp}, for 20 pixels, it
does about 1,550 operations, thus an estimated execution time of
-$1,550/155=$10$\mu $s. For a more realistic evaluation, we constructed
+$1,550/155=$10$\mu$s. For a more realistic evaluation, we constructed
a file of 1Mo containing 200 profiles of 20 pixels, equally
scattered. This file is equivalent to an image stored in a device file
representing the camera. We obtained an average of 10.5$\mu$s by
Nevertheless, the cost of transferring profile in GPU memory and of taking
back results would be prohibitive compared to computation time.
-It should be noticed that when possible, it is more efficient to pipeline the
-computation. For example, supposing that 200 profiles of 20 pixels
-could be pushed sequentially in a pipelined unit cadenced at a 100MHz
-(i.e. a pixel enters in the unit each 10ns), all profiles would be
-treated in $200\times 20\times 10.10^{-9}=$ 40$\mu$s plus the latency
-of the pipeline. Such a solution would be meeting our requirements and
-would be 50 times faster than our C code, and even more compared to
-the LabView version use in \cite{AFMCSEM11}. FPGAs are appropriate for
-such implementation, so they turn out to be the computation units of
-choice to reach our performance requirements. Nevertheless, passing
-from a C code to a pipelined version in VHDL is not obvious at all. It
-can even be impossible because of FPGA hardware constraints. All these
-points are discussed in the following sections.
+It should be noticed that when possible, it is more efficient to
+pipeline the computation. For example, supposing that 200 profiles of
+20 pixels could be pushed sequentially in a pipelined unit cadenced at
+a 100MHz (i.e. a pixel enters in the unit each 10ns), all profiles
+would be treated in $200\times 20\times 10.10^{-9}=$ 40$\mu$s plus the
+latency of the pipeline. Such a solution would be meeting our
+requirements and would be 50 times faster than our C code, and even
+more compared to the LabView version. FPGAs are appropriate for such
+implementation, so they turn out to be the computation units of choice
+to reach our performance requirements. Nevertheless, passing from a C
+code to a pipelined version in VHDL is not obvious at all. It can even
+be impossible because of FPGA hardware constraints. All these points
+are discussed in the following sections.
\section{An hardware/software solution}
\label{sec:solus}
In this section we present parts of the computing solution to the above
-requirements. The hardware part consists in a high-speed camera, linked on an
+requirements. The hardware part consists in a high-speed camera linked on an
embedded board hosting two FPGAs. In this way, the camera output stream can be
pushed directly into the FPGA. The software part is mostly the VHDL code that
deserializes the camera stream, extracts profiles and computes the deflection.
We first give some general information about FPGAs, then we
describe the FPGA board we use for implementation and finally the two
algorithms for phase computation are detailed. Presentation of VHDL
-implementations is postponned until Section \ref{Experimental tests}.
+implementations is postponned until Section \ref{sec:xp-test}.
evolve from state to state. Moreover, HDL instructions can be executed
concurrently. Signals may be combined with basic logic operations to
produce new states that are assigned to another signal. States are mainly expressed as
-arrays of bits. Fortunately, libraries propose some higher levels
+arrays of bits. Fortunately, libraries propose higher levels
representations like signed integers, and arithmetic operations.
Furthermore, even if FPGAs are cadenced more slowly than classic processors,
\subsection{Two algorithms for phase computation}
-In \cite{AFMCSEM11}, $f$ the frequency and $\theta $\ the phase of the
-light wave are computed thanks to spline interpolation. As said in
-section \ref{sec:deflest}, $f$ is computed only once time but the
-phase needs to be computed for each image. This is why, in this paper,
-we focus on its computation.
+As said in section \ref{sec:deflest}, $f$ is computed only once but
+the phase needs to be computed for each image. This is why, in this
+paper, we focus on its computation. The next section describes the
+original method, based on spline interpolation, and section
+\ref{sec:algo-square} presents the new one based on least
+squares. Finally, in section \ref{sec:algo-comp}, we compare the two
+algorithms from their FPGA implementation point of view.
\subsubsection{Spline algorithm (SPL)}
We denote by $M$ the number of pixels in a segment used for phase
computation. For the sake of simplicity of the notations, we consider
-the light intensity I a function on the interval [0,M] which is the
-range of a one-to-one mapping defined on the physical segment. The
-pixels are assumed to be regularly spaced and centered at the
-positions $x^{p}\in\{0,1,\ldots,M-1\}.$ We use the simplest definition
-of a pixel, namely the value of $I$ at its center. The pixel
-intensities are considered as pre-normalized so that their minimum and
-maximum have been resized to $-1$ and $1$.
+the light intensity $I$ a function on the interval [0,M] which itself
+is the range of a one-to-one mapping defined on the physical
+segment. The pixels are assumed to be regularly spaced and centered at
+the positions $x^{p}\in\{0,1,\ldots,M-1\}.$ We use the simplest
+definition of a pixel, namely the value of $I$ at its center. The
+pixel intensities are considered as pre-normalized so that their
+minimum and maximum have been resized to $-1$ and $1$.
The first step consists in computing the cubic spline interpolation of
the intensities. This allows for interpolating $I$ at a larger number
$L=k\times M$ of points (typically $k=4$ is sufficient) $%
x^{s}$ in the interval $[0,M[$. During the precomputation sequence,
-the second step is to determin the affine part $a.x+b$ of $I$. It is
+the second step is to determine the affine part $a.x+b$ of $I$. It is
found with an ordinary least square method, taking account the $L$
points. Values of $I$ in $x^s$ are used to compute its intersections
with $a.x+b$. The period of $I$ (and thus its frequency) is deduced
\textit{Remarks: }
\begin{itemize}
-\item The frequency could also be obtained using the derivates of spline
+\item The frequency could also be obtained using the derivative of spline
equations, which only implies to solve quadratic equations but certainly
yields higher errors.
\end{itemize}
\subsubsection{Least square algorithm (LSQ)}
-
-Assuming that we compute the phase during the acquisition loop, equation \ref%
-{equ:profile} has only 4 parameters: $a,b,A$, and $\theta $, $f$ and $x$
-being already known. Since $I$ is non-linear, a least square method based on
-a Gauss-Newton algorithm can be used to determine these four parameters.
-This kind of iterative process ends with a convergence criterion, so it is
-not suited to our design goals. Fortunately, it is quite simple to reduce
-the number of parameters to only $\theta $. Firstly, the affine part $ax+b$
-is estimated from the $M$ values $I(x^{p})$ to determine the rectified
+\label{sec:algo-square}
+Assuming that we compute the phase during the acquisition loop,
+equation \ref{equ:profile} has only 4 parameters: $a,b,A$, and $\theta
+$, $f$ and $x$ being already known. A least
+square method based on a Gauss-Newton algorithm can be used to
+determine these four parameters. This kind of iterative process ends
+with a convergence criterion, so it is not suited to our design
+goals. Fortunately, it is quite simple to reduce the number of
+parameters to $\theta$ only. Firstly, the affine part $ax+b$ is
+estimated from the $M$ values $I(x^{p})$ to determine the rectified
intensities,%
\begin{equation*}
I^{corr}(x^{p})\approx I(x^{p})-a.x^{p}-b.
\item Lookup tables (namely lut$_{sfi}$ and lut$_{cfi}$ in the following algorithms) can be
set with the $2.M$ values $\sin (2\pi f.i)$ and $\cos (2\pi f.i)$.
-\item A simple method to find a zero $\theta ^{\ast }$ of the optimality
-condition is to discretize the range $[-\pi ,\pi ]$ with a large number $%
-nb_{s}$ of nodes and to find which one is a minimizer in the absolute value
-sense. Hence, three other lookup tables (lut$_{s}$, lut$_{c}$ and lut$_{A}$) can be set with the $%
-3\times nb_{s}$ values $\sin \theta ,$ $\cos \theta ,$
+\item A simple method to find a zero $\theta ^{\ast }$ of the
+ optimality condition is to discretize the range $[-\pi ,\pi ]$ with
+ a large number $%
+ nb_{s}$ of nodes and to find which one is a minimizer in the
+ absolute value sense. Hence, three other lookup tables (lut$_{s}$,
+ lut$_{c}$ and lut$_{A}$) can be set with the $3\times nb_{s}$ values
+ $\sin \theta$, $\cos \theta$, and
\begin{equation*}
\left[ cos2\theta \sum_{i=0}^{M-1}sin(4\pi f.i)+sin2\theta
\sum_{i=0}^{M-1}cos(4\pi f.i)\right] .
\subsubsection{Algorithm comparison}
\label{sec:algo-comp}
-We compared the two algorithms on the base of three criteria:
+We compared the two algorithms regarding three criteria:
\begin{itemize}
-\item precision of results on a cosines profile, distorted by noise,
+\item precision of results on a cosines profile distorted by noise,
\item number of operations,
-\item complexity of FPGA implementation
+\item complexity of FPGA implementation.
\end{itemize}
-For the first item, we produced a matlab version of each algorithm,
+For the first item, we produced a Matlab version of each algorithm,
running in double precision. The profile was generated for about
34,000 different quadruplets of periods ($\in \lbrack 3.1,6.1]$, step
= 0.1), phases ($\in \lbrack -3.1,3.1]$, steps = 0.062) and slopes
($\in \lbrack -2,2]$, step = 0.4). Obviously, the discretization of
$[-\pi ,\pi ]$ introduces an error in the phase estimation. It is at
most equal to $\frac{\pi}{nb_s}$. From some experiments on a $17\times
-4$ array, authors of \cite{AFMCSEM11} noticed an average ratio of 50
+4$ array, we noticed an average ratio of 50
between phase variation in radians and lever end position in
nanometers. Assuming such a ratio and $nb_s = 1024$, the maximum lever
deflection error would be 0.15nm which is smaller than 0.3nm, the best
-precision achieved with the setup used in \cite{AFMCSEM11}.
+precision achieved with the setup used.
Moreover, pixels have been paired and the paired intensities have been
perturbed by addition of a random number uniformly picked in
very close to the best precision.
It is very hard to predict which level of noise will be present in
-real experiments and how it will distort the profiles. Authors of
-\cite{AFMCSEM11} gave us the authorization to exploit some of their
-results on a $17\times 4$ array. It allowed us to compare experimental
-profiles to simulated ones. We can see on figure \ref{fig:noise20} the
-profile with $N=10$ that leads to the biggest error. It is a bit
-distorted, with pikes and straight/rounded portions. In fact, it is
-very close to some of the worst experimental profiles. Figure
-\ref{fig:noise60} shows a sample of worst profile for $N=30$. It is
-completely distorted, largely beyond any experimental ones. Obviously,
-these comparisons are a bit subjective and experimental profiles
-could also be more distorted on other experiments. Nevertheless,
-they give an idea about the possible error.
+real experiments and how it will distort the profiles. Results on
+a $17\times 4$ array allowed us to compare experimental profiles to
+simulated ones. We can see on figure \ref{fig:noise20} the profile
+with $N=10$ that leads to the biggest error. It is a bit distorted,
+with pikes and straight/rounded portions. In fact, it is very close to
+some of the worst experimental profiles. Figure \ref{fig:noise60}
+shows a sample of worst profile for $N=30$. It is completely
+distorted, largely beyond any experimental ones. Obviously, these
+comparisons are a bit subjective and experimental profiles could also
+be more distorted on other experiments. Nevertheless, they give an
+idea about the possible error.
\begin{figure}[ht]
\begin{center}
maximum. So, for larger multiplications, several DSP must be combined
which increases the overall latency.
-In the present algorithms, the hardest constraint does not come from the
-FPGA characteristics but from the algorithms. Their VHDL implementation can
-be efficient only if they can be fully (or near) pipelined. We observe that
-only a small part of SPL can be pipelined, indeed, the computation of spline
-coefficients implies to solve a linear tridiagonal system which matrix and
-right-hand side are computed from incoming pixels intensity but after, the
-back-solve starts with the latest values, which breaks the pipeline.
-Moreover, SPL relies on interpolating far more points than profile size.
-Thus, the end of SPL works on a larger amount of data than at the beginning,
-which also breaks the pipeline.
+Nevertheless, in the present algorithms, the hardest constraint does
+not come from the FPGA characteristics but from the algorithms
+themselves. Their VHDL implementation can be efficient only if they
+can be fully (or near) pipelined. We observe that only a small part of
+SPL can be pipelined, indeed, the computation of spline coefficients
+implies to solve a linear tridiagonal system which matrix and
+right-hand side are computed from incoming pixels intensity but after,
+the back-solve starts with the latest values, which breaks the
+pipeline. Moreover, SPL relies on interpolating far more points than
+profile size. Thus, the end of SPL works on a larger amount of data
+than at the beginning, which also breaks the pipeline.
LSQ has not this problem since all parts, except the dichotomic search, work
on the same amount of data, i.e. the profile size. Furthermore, LSQ requires
\section{VHDL implementation and experimental tests}
-\label{Experimental tests}
+\label{sec:xp-test}
\subsection{VHDL implementation}
by keeping the integer part. For an accurate evaluation of the
division in the computation of $a$ the slope coefficient, we also
scaled the pixel intensities by another power of two. The main problem
-was to determin these factors. Most of the time, they are chosen to
+was to determine these factors. Most of the time, they are chosen to
minimize the error induced by the quantization. But in our case, we
also have some hardware constraints, for example the width and depth of
RAMs or the input size of DSPs. Thus, having a maximum of values that
possibility to extract a new profile from an image every 40
cycles. Nevertheless, it also largely fits our design goals.
-\label{sec:results}
-
\section{Conclusion and perspectives}
In this paper we have presented a full hardware/software solution for
processing chain so that to process real images. Finally, we will address
real-time filtering and control problems for AFM arrays in dynamic regime.
-\section{Acknowledgments}
-We would like to thank A. Meister and M. Favre, from CSEM, for sharing all the
-material we used to write this article and for the time they spent to
-explain us their approach.
+%\section{Acknowledgments}
+%We would like to thank A. Meister and M. Favre, from CSEM, for sharing all the
+%material we used to write this article and for the time they spent to
+%explain us their approach.
\bibliographystyle{plain}
\bibliography{biblio}
