California Institute of Technology, Division of Geological and Planetary Sciences
Seismological Laboratory

Course on Inverse Problems

Albert Tarantola (IPGP, Paris). Teaching Assistant: Ting Chen <tchen@gps.caltech.edu>.

 
March - April - May, 2007

Photos of the students
  • The lecture notes of this course are constituted by my book Inverse Problem Theory (SIAM, 2005). To download the book, click here. There is a second book in preparation, Mapping of Probabilities, with much heavier mathematics. You don't need to read this text (that is very preliminary and evolves in a daily basis). Those who feel curious can have a look at the different parts of the manuscript: Table of Contents, Chapter 1, Chapter 2, Chapter 3, Chapter 4, Appendices, Bibliography.
  • The introduction to the course is at the bottom of this page. Please, read also this text (qualitative and short).


Program: (the complete program is here [pdf document])
  • Basic notions of set theory.
  • Basic notions of probability theory.
  • Models, Observations, the Forward Simulation Problem.
  • The Inverse Simulation Problem (the Popper-Bayes approach).
  • Explicit use of probabilities.
  • Monte Carlo methods.
  • Optimization methods (least-squares, least-absolute values, ...).
  • Functional least-squares.
  • Examples.



Solved exercises (with Mathematica codes):

To prepare this teaching at CalTech, I have decided to collect the exercices that I have been solving in class during the last years, to try putting them under the form of a book. It is not yet clear that I will succeed, and your help may be crucial. This tentative text and the exercices are available from this link. I do not suggest that you download the entire text (it is too preliminary): at the end of each lesson, I will suggest which section you should download and read to prepare for the next lesson. Although I know reasonably well Mathematica, I am not very familiar with Matlab and Scilab, so I also may need your help in translating some of the codes.



Introduction:

Physical theories allow us to make predictions: given a complete description of a physical system, we can predict the outcome of some measurements. This problem of predicting the result of measurements is called the modelization problem, the simulation problem, or the forward problem. The inverse problem consists of using the actual result of some measurements to infer the values of the parameters that characterize the system.


While the forward problem has (in deterministic physics) a unique solution, the inverse problem does not. As an example, consider measurements of the gravity field around a planet: given the distribution of mass inside the planet, we can uniquely predict the values of the gravity field around the planet (forward problem), but there are different distributions of mass that give exactly the same gravity field in the space outside the planet. Therefore, the inverse problem —of inferring the mass distribution from observations of the gravity field— has multiple solutions (in fact, an infinite number).

Because of this, in the inverse problem, one needs to make explicit any available a priori information on the model parameters. One also needs to be careful in the representation of the data uncertainties.

The most general (and simple) theory is obtained when using a probabilistic point of view, where the a priori information on the model parameters is represented by a probability distribution over the "model space." The theory developed here explains how this a priori probability distribution is transformed into the a posteriori probability distribution, by incorporating a physical theory (relating the model parameters to some observable parameters) and the actual result of the observations (with their uncertainties).

To develop the theory, we shall need to examine the different types of parameters that appear in physics and to be able to understand what a total absence of a priori information on a given parameter may mean.

Although the notion of the inverse problem could be based on conditional probabilities and Bayes's theorem, I choose to introduce a more general notion, that of the "combination of states of information," that is, in principle, free from the special difficulties appearing in the use of conditional probability densities (like the well-known Borel paradox).

The general theory has a simple (probabilistic) formulation and applies to any kind of inverse problem, including linear as well as strongly nonlinear problems. Except for very simple examples, the probabilistic formulation of the inverse problem requires a resolution in terms of "samples" of the a posteriori probability distribution in the model space. This, in particular, means that the solution of an inverse problem is not a model but a collection of models (that are consistent with both the data and the a priori information). This is why Monte Carlo (i.e., random) techniques are examined in this course. With the increasing availability of computer power, Monte Carlo techniques are being increasingly used.

Some special problems, where nonlinearities are weak, can be solved using special, very efficient techniques that do not differ essentially from those used, for instance, by Laplace in 1799, who introduced the "least-absolute-values" and the "minimax" criteria for obtaining the best solution, or by Legendre in 1801 and Gauss in 1809, who introduced the "least-squares" criterion.

The first part of the course deals exclusively with discrete inverse problems with a finite number of parameters. Some real problems are naturally discrete, while others contain functions of a continuous variable and can be discretized if the functions under consideration are smooth enough compared to the sampling length, or if the functions can conveniently be described by their development on a truncated basis. The advantage of a discretized point of view for problems involving functions is that the mathematics is easier. The disadvantage is that some simplifications arising in a general approach can be hidden when using a discrete formulation. (Discretizing the forward problem and setting a discrete inverse problem is not always equivalent to setting a general inverse problem and discretizing for the practical computations.)

The second part of the course deals with general inverse problems, which may contain such functions as data or unknowns. As this general approach contains the discrete case in particular, the separation into two parts corresponds only to a didactical purpose.

Although this course contains a lot of mathematics, it does not require any special mathematical background, as all the notions shall be explicitly introduced. The basic objective is to explain how a method of acquisition of information can be applied to the actual world, and many of the arguments are heuristic.

The course is based on the book Inverse Problem Theory and Methods for Model Parameter Estimation (Tarantola, 2005), whose electronic form is freely available for downdoad (see the link above).

Links to: Division of Geological and Planetary Sciences (Pasadena), IPG (Paris)