- The lecture notes of this course are constituted
by my book Inverse
Problem Theory (SIAM, 2005). To download the book, click
here. There is a second book
of Probabilities, with much heavier mathematics.
You don't need to read this text (that is very
preliminary and evolves in a daily basis). Those who feel curious can
have a look at the different parts of the manuscript: Table
- The introduction
to the course is at
the bottom of this page. Please, read also this
- Lessons: Lesson I (screen, printer). Lesson II (screen, paper). Lesson III (screen, paper). Lesson IV (screen, paper). Lesson V (screen, paper). Lesson VI (screen, paper). Lesson VII (screen, paper). Lesson VIII (screen, paper). Lesson IX (screen, paper). Lesson X (screen, paper). Lesson XI (screen, paper). Lesson XII (screen, paper). Lesson XIII (screen, paper). Lesson XIV
paper). Lesson XV (screen, paper).
Lesson XVI (screen, paper). Lesson XVII (screen).
(the complete program
- Basic notions of set theory.
- Basic notions of probability theory.
- Models, Observations, the Forward Simulation Problem.
- The Inverse Simulation Problem (the Popper-Bayes approach).
- Explicit use of probabilities.
- Monte Carlo methods.
- Optimization methods (least-squares, least-absolute values, ...).
- Functional least-squares.
exercises (with Mathematica
To prepare this teaching at CalTech,
I have decided to collect the exercices that I have been solving in
class during the last years, to try putting them under the form of a
book. It is not yet clear that I will succeed, and your help may be
crucial. This tentative text and the exercices are available from this link.
I do not suggest that you download the entire text (it is too
preliminary): at the end of each lesson, I will suggest which section
you should download and read to prepare for the next lesson. Although I
know reasonably well Mathematica,
I am not very familiar with Matlab
and Scilab, so I also may
need your help in translating some of the codes.
theories allow us to make
predictions: given a complete description of a physical system, we can
predict the outcome of some measurements. This problem of predicting
the result of measurements is called the modelization problem, the simulation problem, or the forward problem. The inverse problem consists of using
the actual result of some measurements to infer the values of the
parameters that characterize the system.
While the forward problem has (in deterministic physics) a
solution, the inverse problem does not. As an example, consider
measurements of the gravity field around a planet: given the
distribution of mass inside the planet, we can uniquely predict the
values of the gravity field around the planet (forward problem), but
there are different distributions of mass that give exactly the same gravity field in
the space outside the planet. Therefore, the inverse problem —of
inferring the mass distribution from observations of the gravity field—
has multiple solutions (in fact, an infinite number).
Because of this, in the inverse problem, one needs to make
available a priori information on the model parameters. One also needs
to be careful in the representation of the data uncertainties.
The most general (and simple) theory is obtained when using a
probabilistic point of view, where the a priori information on the
model parameters is represented by a probability distribution over the
"model space." The theory developed here explains how this a priori
probability distribution is transformed into the a posteriori
probability distribution, by incorporating a physical theory (relating
the model parameters to some observable parameters) and the actual
result of the observations (with their uncertainties).
To develop the theory, we shall need to examine the different
parameters that appear in physics and to be able to understand what a
total absence of a priori information on a given parameter may mean.
Although the notion of the inverse problem could be based on
conditional probabilities and Bayes's theorem, I choose to introduce a
more general notion, that of the "combination of states of
information," that is, in principle, free from the special difficulties
appearing in the use of conditional probability densities (like the
well-known Borel paradox).
The general theory has a simple (probabilistic) formulation and
to any kind of inverse problem, including linear as well as strongly
nonlinear problems. Except for very simple examples, the probabilistic
formulation of the inverse problem requires a resolution in terms of
"samples" of the a posteriori probability distribution in the model
space. This, in particular, means that the solution of an inverse
problem is not a model but a collection of models (that are consistent
with both the data and the a priori information). This is why Monte
Carlo (i.e., random) techniques are examined in this course. With the
increasing availability of computer power, Monte Carlo techniques are
being increasingly used.
Some special problems, where nonlinearities are weak, can be
using special, very efficient techniques that do not differ essentially
from those used, for instance, by Laplace in 1799, who introduced the
"least-absolute-values" and the "minimax" criteria for obtaining the
best solution, or by Legendre in 1801 and Gauss in 1809, who introduced
the "least-squares" criterion.
The first part of the course deals exclusively with discrete
problems with a finite number
of parameters. Some real problems are naturally discrete, while others
contain functions of a continuous variable and can be discretized if
the functions under consideration are smooth enough compared to the
sampling length, or if the functions can conveniently be described by
their development on a truncated basis. The advantage of a discretized
point of view for problems involving functions is that the mathematics
is easier. The disadvantage is that some simplifications arising in a
general approach can be hidden when using a discrete formulation.
(Discretizing the forward problem and setting a discrete inverse
problem is not always equivalent to setting a general inverse problem
and discretizing for the practical computations.)
The second part of the course deals with general inverse
which may contain such functions as data or unknowns. As this general
approach contains the discrete case in particular, the separation into
two parts corresponds only to a didactical purpose.
Although this course contains a lot of mathematics, it does not
any special mathematical background, as all the notions shall be
explicitly introduced. The basic objective is to explain how a method
of acquisition of information can be applied to the actual world, and
many of the arguments are heuristic.
The course is based on the book Inverse
Problem Theory and Methods for
Model Parameter Estimation (Tarantola, 2005), whose electronic
freely available for downdoad (see the link above).