[book_chic.git] / chapter2.tex

%%%%%%%%%%%%%%%%%%%%% chapter.tex %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% sample chapter
%
% Use this file as a template for your own input.
%
%%%%%%%%%%%%%%%%%%%%%%%% Springer-Verlag %%%%%%%%%%%%%%%%%%%%%%%%%%
%\motto{Use the template \emph{chapter.tex} to style the various elements of your chapter content.}
\chapter{From the founding situations of the SIA to its formalization}
\label{intro} % Always give a unique label
% use \chaptermark{}
% to alter or adjust the chapter heading in the running head



\abstract{
Starting from mathematical didactic situations, the implicitative
statistical analysis method develops as problems are encountered and
questions are asked.
Its main objective is to structure data crossing subjects and
variables, to extract inductive rules between variables and, based on
the contingency of these rules, to explain and therefore forecast in
various fields: psychology, sociology, biology, etc.
It is for this purpose that the concepts of intensity of implication,
class cohesion, implication-inclusion, significance of hierarchical
levels, contribution of additional variables, etc., are based.
Similarly, the processing of binary variables (e.g., descriptors) is
gradually being supplemented by the processing of modal, frequency
and, recently, interval and fuzzy variables.
}

\section{Preamble}

Human operative knowledge is mainly composed of two components: that
of facts and that of rules between facts or between rules themselves.
It is his learning that, through his culture and his personal
experiences, allows him to gradually develop these forms of knowledge,
despite the regressions, the questioning, the ruptures that arise at
the turn of decisive information.
However, we know that these dialectically contribute to ensuring a
balanced operation.
However, the rules are inductively formed in a relatively stable way
as soon as the number of successes, in terms of their explanatory or
anticipatory quality, reaches a certain level (of confidence) from
which they are likely to be implemented.
On the other hand, if this (subjective) level is not reached, the
individual's economy will make him resist, in the first instance, his
abandonment or criticism.
Indeed, it is costly to replace the initial rule with another rule
when a small number of infirmations appear, since it would have been
reinforced by a large number of confirmations.
An increase in this number of negative instances, depending on the
robustness of the level of confidence in the rule, may lead to its
readjustment or even abandonment.
Laurent Fleury~\cite{Fleury}, in his thesis, correctly cites the
example - which Régis repeats - of the highly admissible rule: "all
Ferraris are red".
This very robust rule will not be abandoned when observing a single or
two counter-examples.
Especially since it would not fail to be quickly
re-comforted.

Thus, contrary to what is legitimate in mathematics, where not all
rules (theorem) suffer from exception, where determinism is total,
rules in the human sciences, more generally in the so-called "soft"
sciences, are acceptable and therefore operative as long as the number
of counter-examples remains "bearable" in view of the frequency of
situations where they will be positive and effective.
The problem in data analysis is then to establish a relatively
consensual numerical criterion to define the notion of a level of
confidence that can be adjusted to the level of requirement of the
rule user.
The fact that it is based on statistics is not surprising.
That it has a property of non-linear resistance to noise (weakness of
the first counter-example(s)) may also seem natural, in line with the
"economic" meaning mentioned above.
That it collapses if counter-examples are repeated also seems to have
to guide our choice in the modeling of the desired criterion.
This text presents the epistemological choice we have made.
As such it is therefore refutable, but the number of situations and
applications where it has proved relevant and fruitful leads us to
reproduce its genesis here.

\section{Introduction}

Different theoretical approaches have been adopted to model the
extraction and representation of imprecise (or partial) inference
rules between binary variables (or attributes or characters)
describing a population of individuals (or subjects or objects).
But the initial situations and the nature of the data do not change
the initial problem.
It is a question of discovering non-symmetrical inductive rules to
model relationships of the type "if a then almost b".
This is, for example, the option of Bayesian networks~\cite{Amarger}
or Galois lattices~\cite{Simon}.
But more often than not, however, since the correlation and the
${\chi}^2$ test are unsuitable because of their symmetric nature,
conditional probability~\cite{Loevinger, Agrawal,Grasn}  remains the
driving force behind the definition of the association, even when the
index of this selected association is multivariate~\cite{Bernard}.



Moreover, to our knowledge, on the one hand, most often the different
and interesting developments focus on proposals for a partial
implication index for binary data~\cite{Lermana} or \cite{Lallich}, on
the other hand, this notion is not extended to other types of
variables, to extraction and representation according to a rule graph
or a hierarchy of meta-rules; structures aiming at access to the
meaning of a whole not reduced to the sum of its
parts~\cite{Seve}\footnote{This is what the philosopher L. Sève
  emphasizes :"... in the non-additive, non-linear passage of the
  parts to the whole, there are properties that are in no way
  precontained in the parts and which cannot therefore be explained by
  them" }, i.e. operating as a complex non-linear system.
For example, it is well known, through usage, that the meaning of a
sentence does not completely depend on the meaning of each of the
words in it (see the previous chapter, point 4).

Let us return to what we believe is fertile in the approach we are
developing.
It would seem that, in the literature, the notion of implication index
is also not extended to the search for subjects and categories of
subjects responsible for associations.
Nor that this responsibility is quantified and thus leads to a
reciprocal structuring of all subjects, conditioned by their
relationships to variables.
We propose these extensions here after recalling the founding
paradigm.


\section{Implication intensity in the binary case}

\subsection{Fundamental and founding situation}

A set of objects or subjects E is crossed with variables
(characters, criteria, successes,...) which are interrogated as
follows: "to what extent can we consider that instantiating variable\footnote{Throughout the book, the word "variable" refers to both an isolated variable in premise (example: "to be blonde") or a conjunction of isolated variables (example: "to be blonde and to be under 30 years old and to live in Paris")} $a$
implies instantiating variable $b$?
In other words, do the subjects tend to be $b$ if we know that they are
$a$?".
In natural, human or life sciences situations, where theorems (if $a$
then $b$) in the deductive sense of the term cannot be established
because of the exceptions that taint them, it is important for the
researcher and the practitioner to "mine into his data" in order to
identify sufficiently reliable rules (kinds of "partial theorems",
inductions) to be able to conjecture\footnote{"The exception confirms the rule", as the popular saying goes, in the sense that there would be no exceptions if there were no rule} a possible causal relationship,
a genesis, to describe, structure a population and make the assumption
of a certain stability for descriptive and, if possible, predictive
purposes.
But this excavation requires the development of methods to guide it
and to free it from trial and error and empiricism.


\subsection{Mathematization}

To do this, following the example of the I.C. Lerman similarity
measurement method \cite{Lerman,Lermanb}, following the classic
approach in non-parametric tests (e. g. Fischer, Wilcoxon, etc.), we
define~\cite{Grasb,Grasf} the confirmatory quality measure of the
implicative relationship $a \Rightarrow b$ from the implausibility of
the occurrence in the data of the number of cases that invalidate it,
i.e. for which $a$ is verified without $b$ being verified. This
amounts to comparing the difference between the quota and the
theoretical if only chance occurred\footnote{"...[in agreement with
    Jung] if the frequency of coincidences does not significantly
  exceed the probability that they can be calculated by attributing
  them solely by chance to the exclusion of hidden causal
  relationships, we certainly have no reason to suppose the existence
  of such relationships.", H. Atlan~\cite{Atlana}}.
But when analyzing data, it is this gap that we take into account and
not the statement of a rejection or null hypothesis eligibility.
This measure is relative to the number of data verifying $a$ and not
$b$ respectively, the circumstance in which the involvement is
precisely put in default.
It quantifies the expert's "astonishment" at the unlikely small number
of counter-examples in view of the supposed independence between the
variables and the numbers involved.

Let us be clear. A finite set $V$ of $v$ variables is given: $a$, $b$,
$c$,...
In the classical paradigmatic situation and initially retained, it is
about the performance (success-failure) to items of a questionnaire.
To a finite set $E$ of $n$ subjects $x$, functions of the type : $x
\rightarrow a(x)$ where $a(x) = 1$ (or $a(x) = true$) if $x$ satisfies
or has the character $a$ and $0$ (or $a(x) = false$) otherwise are
associated by abuse of writing.
In artificial intelligence, we will say that $x$ is an example or an
instance for $a$ if $a(x) = 1$ and a counter-example if not.


The $a \Rightarrow b$ rule is logically true if for any $x$ in the
sample, $b(x)$ is null only if $a(x)$ is also null; in other words if
set $A$ of the $x$ for which $a(x)=1$ is contained in set $B$ of the
$x$ for which $b(x)=1$.
However, this strict inclusion is only exceptionally observed in the
pragmatically encountered experiments.
In the case of a knowledge questionnaire, we could indeed observe a
few rare students passing an item $a$ and not passing item $b$,
without contesting the tendency to pass item $b$ when we have passed
item $a$.
With regard to the cardinals of $E$ (of size $n$), but also of $A$ (or
$n_a$) and $B$ (or $n_b$), it is therefore the "weight" of the
counter-examples (or) that must be taken into account in order to
statistically accept whether or not to keep the quasi-implication or
quasi-rule  $a \Rightarrow b$.  Thus, it is from the dialectic of
example-counter-examples that the rule appears as the overcoming of
contradiction.

\subsection{Formalization}

To formalize this quasi-rule, we consider any two parts $X$ and $Y$ of
$E$, chosen randomly and independently (absence of a priori link
between these two parts) and of the same respective cardinals as $A$
and $B$. Let $\overline{Y}$ and $\overline{B}$ be the respective complementary of $Y$ and $B$ in $E$ of the same cardinal $n_{\overline{b}}= n-n_b$.

We will then say:

\definition $a \Rightarrow b$ is acceptable at confidence level
$1-\alpha$ if and only if
$$Pr[Card(X\cap \overline{Y})\leq card(A\cap \overline{B})]\leq \alpha$$

\begin{figure}[htbp]
  \centering
\includegraphics[scale=0.34]{chap2fig1.png}
 \caption{The dark grey parts correspond to the counter-examples of the
   implication $a \Rightarrow b$}
\label{chap2fig1}      
\end{figure}

It is established \cite{Lermanb} that, for a certain drawing process,
the random variable $Card(X\cap \overline{Y})$ follows the Poisson law
of parameter $\frac{n_a n_{\overline{b}}}{n}$.
We achieve this same result by proceeding differently in the following
way:

Note $X$ (resp. $Y$) the random subset of binary transactions where
$a$ (resp. $b$) would appear, independently, with the frequency
$\frac{n_a}{n}$ (resp. $\frac{n_b}{n}$).
To specify how the transactions specified in variables $a$ and $b$,
respectively $A$ and $B$, are extracted, for example, the following
semantically permissible assumptions are made regarding the
observation of the event: $[a=1~ and~ b=0]$. $(A\cap
\overline{B})$\footnote{We then note $\overline{v}$ the variable
  negation of $v$  (or $not~ v$) and $\overline{P}$ the complementary
  part of the part P of E.} is the subset of transactions,
counter-examples of implication $a \Rightarrow b$: 

Assumptions:
\begin{itemize}
\item h1: the waiting times of an event $[a~ and~ not~ b]$ are independent
  random variables;
\item h2: the law of the number of events occurring in the time
  interval $[t,~ t+T[$ depends only on T;
\item h3: two such events cannot occur simultaneously
\end{itemize}

It is then demonstrated (for example in~\cite{Saporta}) that the
number of events occurring during a period of fixed duration $n$
follows a Poisson's law of parameter $c.n$ where $c$ is called the
rate of the apparitions process during the unit of time.


However, for each transaction assumed to be random, the event $[a=1]$
has the probability of the frequency $\frac{n_a}{n}$, the event[b=0]
has as probability the frequency, therefore the joint event $[a=1~
  and~ b=0]$ has for probability estimated by the frequency
$\frac{n_a}{n}. \frac{n_{\overline{b}}}{b}$ in the hypothesis of absence of an a priori link between a and b (independence).

We can then estimate the rate $c$ of this event by $\frac{n_a}{n}. \frac{n_{\overline{b}}}{b}$.

Thus for a duration of time $n$, the occurrences of the event $[a~ and~ not~b]$ follow a Poisson's law of parameter : 
$$\lambda = \frac{n_a.n_{\overline{b}}}{n}$$

As a result, $Pr[Card(X\cap \overline{Y})= s]= e^{-\lambda}\frac{\lambda^s}{s!}$

Consequently, the probability that the hazard will lead, under the
assumption of the absence of an a priori link between $a$ and $b$, to
more counter-examples than those observed is:

$$Pr[Card(X\cap \overline{Y})\leq card(A\cap \overline{B})] =
\sum^{card(A\cap \overline{B})}_{s=0}  e^{-\lambda}\frac{\lambda^s}{s!} $$

 But other legitimate drawing processes lead to a binomial law, or
 even a hypergeometric law (itself not semantically adapted to the
 situation because of its symmetry). Under suitable convergence
 conditions, these two laws are finally reduced to the Poisson Law
 above (see Annex to this chapter).
 
If $n_{\overline{b}}\neq 0$, we reduce and center this Poison variable
into the variable:

$$Q(a,\overline{b})= \frac{card(X \cap \overline{Y})) -  \frac{n_a.n_{\overline{b}}}{n}}{\sqrt{\frac{n_a.n_{\overline{b}}}{n}}}  $$

In the experimental realization, the observed value of
$Q(a,\overline{b})$ is $q(a,\overline{b})$.
It estimates a gap between the contingency $(card(A\cap
\overline{B}))$ and the value it would have taken if there had been
independence between $a$ and $b$.

\definition $$q(a,\overline{b}) = \frac{n_{a \wedge \overline{b}}-  \frac{n_a.n_{\overline{b}}}{n}}{\sqrt{\frac{n_a.n_{\overline{b}}}{n}}}$$
is called the implication index, the number used as an indicator of
the non-implication of $a$ to $b$.
In cases where the approximation is properly legitimized (for example
$\frac{n_a.n_{\overline{b}}}{n}\geq 4$), the variable
$Q(a,\overline{b})$ approximately follows the reduced centered normal
distribution. The intensity of implication, measuring the quality of
$a\Rightarrow b$, for $n_a\leq n_b$ and  $nb \neq n$, is then defined
from the index $q(a,\overline{b})$ by:

\definition 
The implication intensity  that measures the inductive quality of a
over b is:
$$\varphi(a,b)=1-Pr[Q(a,\overline{b})\leq q(a,\overline{b})] =
\frac{1}{\sqrt{2 \pi}} \int^{\infty}_{ q(a,\overline{b})}
e^{-\frac{t^2}{2}} dt,~ if~ n_b \neq n$$
$$\varphi(a,b)=0,~ otherwise$$
As a result, the definition of statistical implication becomes:
\definition 
Implication  $a\Rightarrow b$ is admissible at confidence level
$1-\alpha $ if and only if: 
$$\varphi(a,b)\geq 1-\alpha$$


It should be recalled that this modeling of quasi-implication measures
the astonishment to note the smallness of counter-examples compared to
the surprising number of instances of implication.
It is a measure of the inductive and informative quality of
implication. Therefore, if the rule is trivial, as in the case where
$B$ is very large or coincides with $E$, this astonishment becomes
small.
We also demonstrate~\cite{Grasf} that this triviality results in a
very low or even zero intensity of implication: If, $n_a$ being fixed
and $A$ being included in $B$, $n_b$ tends towards $n$ ($B$ "grows"
towards $E$), then $\varphi(a,b)$ tends towards $0$. We therefore
define, by "continuity":$\varphi(a,b) = 0$ if $n_b = n$. Similarly, if
$A\subset B$, $\varphi(a,b)$ may be less than $1$ in the case where
the inductive confidence, measured by statistical surprise, is
insufficient.

{\bf \remark Total correlation, partial correlation}


We take here the notion of correlation in a more general sense than
that used in the domain that develops the linear correlation
coefficient (linear link measure) or the correlation ratio (functional
link measure).
In our perspective, there is a total (or partial) correlation between
two variables $a$ and $b$ when the respective events they determine
occur (or almost occur) at the same time, as well as their opposites.
However, we know from numerical counter-examples that correlation and
implication do not come down to each other, that there can be
correlation without implication and vice versa~\cite{Grasf} and below.
If we compare the implication coefficient and the linear correlation
coefficient algebraically, it is clear that the two concepts do not
coincide and therefore do not provide the same
information\footnote{"More serious is the logical error inferred from
  a correlation found to the existence of a causality" writes Albert
  Jacquard in~\cite{Jacquard}, p.159. }.

The quasi-implication of non-symmetric index $q(a,\overline{b})$ does
not coincide with the correlation coefficient $\rho(a, b)$ which is
symmetric and which reflects the relationship between variables a and
b. Indeed, we show~\cite{Grasf} that if $q(a,\overline{b}) \neq 0$
then
$$\frac{\rho(a,b)}{q(a,\overline{b})} = \sqrt{\frac{n}{n_b
    n_{\overline{a}}}} q(a,\overline{b})$$
With the correlation considered from the point of view of linear
correlation, even if correlation and implication are rather in the
same direction, the orientation of the relationship between two
variables is not transparent because it is symmetrical, which is not
the bias taken in the SIA.
From a statistical relationship given by the correlation, two opposing
empirical propositions can be deduced.

The following dual numerical situation clearly illustrates this:


\begin{table}[htp]
\center
\begin{tabular}{|l|c|c|c|}\hline
\diagbox[width=4em]{$a_1$}{$b_1$}&
  1 & 0 & marge\\ \hline
  1 & 96 & 4& 100 \\ \hline
  0 & 50 & 50& 100 \\ \hline
  marge & 146 & 54& 200 \\ \hline
\end{tabular} ~ ~ ~ ~ ~ ~ ~ \begin{tabular}{|l|c|c|c|}\hline
\diagbox[width=4em]{$a_2$}{$b_2$}&
  1 & 0 & marge\\ \hline
  1 & 94 & 6& 100 \\ \hline
  0 & 52 & 48& 100 \\ \hline
  marge & 146 & 54& 200 \\ \hline
\end{tabular}

\caption{Numeric example of difference between implication and
  correlation}
\label{chap2tab1}
\end{table}

In Table~\ref{chap2tab1}, the following correlation and implications
can be computed:\\
Correlation $\rho(a_1,b_1)=0.468$, Implication
$q(a_1,\overline{b_1})=-4.082$\\
Correlation $\rho(a_2,b_2)=0.473$, Implication  $q(a_2,\overline{b_2})=-4.041$


Thus, we observe that, on the one hand, $a_1$ and $b_1$ are less
correlated than $a_2$ and $b_2$ while, on the other hand, the
implication intensity of $a_1$ over $b_1$ is higher than that of $a_2$
over $b_2$ since $q1 <q2$.

On this subject, Alain Ehrenberg in~\cite{Ehrenberg} writes: "The
finding of a correlation does not remove the ambiguity between" when I do $X$, my brain is in state $Y$" and "if I do $X$, it is because my brain is in state $Y$", that is, between something that happens in my brain when I do an action.

\remark  Remember that we consider not only conjunctions of variables
of the type "$a$ and $b$" but also disjunctions such as "($a$ and $b$)
or $c$..." in order to model phenomena that are concepts as it is done
in learning or in artificial intelligence.
The associated calculations remain compatible with the logic of the
proposals linked by connectors.

\remark Unlike the Loevinger Index~\cite{Loevinger}  and conditional
probability $(Pr[B/A])=1$ and all its derivatives, the implication
intensity varies, non-linearly, with the expansion of sets $E$, $A$
and $B$ and weakens with triviality (see Definition 2.3).
Moreover, it
is resistant to noise, especially around $0$ for, which can only make
the relationship we want to model and establish statistically
credible.
Finally, as we have seen, the inclusion of $A$ in $B$ does not ensure
maximum intensity, the inductive quality may not be strong, whereas
$Pr[B/A]$ is equal to $1$~\cite{Grasm,Guillet}.
In paragraph 5, we study more closely the problem of the sensitivity
and stability of the implication index as a function of small
variations in the parameters involved in the study of its
differential.