Grant RFBR 02-01-00093, NSh.2251.2003.1
.
KANTOROVICH METRIC: INITIAL HISTORY AND LITTLE-KNOWN APPLICATIONS
A.Vershik
Mathematical Institute of Russian Ac.Sci. St.Petersburg branch, Fontanka 27, St.Petersburg, 191023, Russia. E-mail address : vershik@pdmi.ras.ru
-
Abstract.
We recall the history of the transportation (Kantorovich) metric and the Monge–Kantorovich problem. We also describe several little-known applications: the first one concerns the theory of decreasing sequences of partitions (tower of measures and iterated metric), the second one relates to Ornstein's theory of Bernoulli automorphisms (
-metric), and the third one is the formulation of the strong Monge–Kantorovich problem in terms of matrix distributions.
Bibliography:
titles.
1 Introduction: the first papers on the transportation problem
The studies on the transportation problem could be called a true pearl in the extremely rich scientific legacy of L. V. Kantorovich. The beauty and naturalness of the formulation, the fundamental character of the main theorem (optimality criterion), and, finally, the wealth of applications (some of them are realized, but new applications keep on arising in areas that appear only now) – all this allows us to place these studies among the classic mathematical works of the 20th century.
Undoubtedly, the same words can be applied to the whole series of papers on linear programming (from which the transportation problem cannot be separated), which became the starting point for further studies on mathematical economics, but here we will only dwell on the remarkable role of what was later called the “Monge–Kantorovich problem” and “transportation metric.” 1
In this introduction we do not intend to give a survey of this huge subject; we will mention only the very first papers of L.V. and his co-authors.
Apparently, L.V. conceived the formulation of the transportation problem soon after he defined the general model of the production planning problem, i.e., in the late 30s (the booklet [8]). However, if we judge from the date of the first publication, the transportation problem was born in 1942, with the publication of the note [10], which later became famous. The year itself predetermined the long road this paper had to walk to become known to specialists. The paper contains an explicit formulation of the general continuous transportation problem on a compact metric space, the dual problem, and the optimality criterion. Later, in the small note [11] published in Uspekhi Mat. Nauk, Kantorovich established a relation to Monge's problem of excavations and embankments, i.e., to the transportation problem on the Euclidean plane. Since then, the general Kantorovich problem is sometimes called the Monge–Kantorovich problem (MK-problem for short). The next paper [6] , joint with a pupil of L.V., M. K. Gavurin, was addressed rather to applied mathematicians and economists; it contained a development of the method of potentials (a version of the method of resolving multipliers suggested by L.V. in 1939) for solving the finite-dimensional transportation problem. Written long before publication, it appeared only in 1949, and this delay was caused not by the wartime conditions, but by the Soviet practice of that time, when each scientific paper that even slightly touched economic (not to mention socio-economic) problems had to go through long and absurd censorship; besides, the paper was published not in a journal, but in a special hard-to-reach volume.
Till 1956, i.e., during 18 years of existence of the new mathematical economic theory, L.V. and his co-authors published less than 10 papers on this subject (I remember G. Sh. Rubinshtein making up, at my request, the complete list of these papers in autumn 1956). Surely, not because texts dealing with these problems were not written. L.V. had already prepared a whole book on economics, whose destiny is an exact and gloomy illustration of the system's attitude to scientific studies that do not keep within obligatory schemes, rigid and hence fruitless. A revised version of the book was not published till almost twenty years later ([13]).
In 1955–56, L.V. decided to “open” this topic; he began to give public and special lectures, to popularize his theory. The moment was chosen quite well. However, the wide distribution and acknowledgment of these studies were still a long way off. One can read about all these events in the book [16] (in particular, in my paper [26]), but a detailed account of the whole story is still to be written.
Let us return to the transportation problem. The third important paper on this subject was the paper [15] by L.V. and his pupil and co-author G. Sh. Rubinshtein. It is this paper that contained an explicit definition of the norm in the space of measures related to the transportation metric. The main observation was that the conjugate space to the space of measures with this norm is the space of Lipschitz functions, and the optimality criterion is nothing more than the dual definition of the norm as a supremum over the sphere of the conjugate space. Before this paper it was not known whether the space of Lipschitz functions is conjugate to any Banach space. At that time (1956–57) I was interested in mathematical economics and maintained close contacts with L.V. and G. Sh. Rubinshtein, and G. Sh. described me in detail the stages of their work; in particular, he said that L.V. was very satisfied by this interpretation of the transportation problem. After this paper, the metric is often called the Kantorovich–Rubinshtein metric.
Here it is worthwhile to make two remarks. Of course, the idea of duality was contained from the very beginning both in the booklet of 1939 (the method of resolving multipliers) and in the note by L.V. in Doklady Akad. Nauk SSSR [9] – the first paper devoted to comprehending the relations between functional analysis and nonclassical linear extremal problems (calculation of norms and extrema); it is worth noting that this was one more example showing the utility of functional analysis for applications; see the paper [14], devoted to applications of linear programming to computational mathematics, and the classical work by L.V. on the Newton method [12]. On the other hand, the technique that consists in taking the objective function as the norm in the space of right-hand sides of an extremal problem (exactly this was suggested in [15]) can be successfully applied to many extremal problems (see, for example, [19, 28]). It was noted more than once that both classics of mathematical economics of the 20th century – von Neumann and Kantorovich – came from functional analysis.
We cannot but mention that eventually, in course of development of the theory of nonclassical extremal problems, other relations became obvious: to the theory of linear inequalities and separability theory, Chebyshev approximations and Krein's
-moment problem, Weyl's studies on convex polytopes and convex geometry as a whole, Bourbaki's theory of polars and combinatorics, etc. 2
Today we would include in this list “tropical” mathematics, or max-plus algebra and impetuious developement of the applications to differential equations, in particualr to Monge-Ampere equation, hydrodynamics and so on (see references). We will not discuss here those illuminated applications.
2 Basic definitions
The transportation problem has been always holding a prominent position among all problems of linear programming due to its general formulation and methods of solution. In what follows, I would like to present several little-known applications of the transportation metric; but first let us recall the formulation of the transportation problem.
Definition 1.
Let
be a compact metric space, and let
and
be two probability Borel measures on
. Consider the Monge–Kantorovich variational problem (MK-problem for short):
set
where
runs over all Borel measures on
with marginal measures
and
.
The quantity
determines a metric on the simplex
of all probability measures on the compact space
; it is called the Kantorovich (or transportation) metric ([10]).
Remark. The measure
is a “plan of transportation” of the distribution
to the distribution
; the integral means the cost of a given transportation plan, and the infimum (the Kantorovich metric) is achieved at the optimal plan.
Theorem 1.
(Kantorovich–Rubinshtein [15]) (1) Consider the vector space
of all (not necessarily positive) Borel measures
with zero charge and finite variation (i.e., the positive part,
, and the negative part,
, of
have the same finite variation) and define the Kantorovich–Rubinshtein norm
of an element
as the Kantorovich distance between the positive and negative parts of
:
Then the space of Lipschitz (up to additive constant) functions with the Lipschitz norm is the conjugate normed space to the space
with the norm
.
(2) A plan
in (1) is optimal if and only if there exists a Lipschitz function
with Lipschitz constant
such that
almost everywhere with respect to the plan
.
We will omit the index
in the notation
if the metric
is fixed, as well as the index
in the notation
.
Remark 1. The Kantorovich metric induces the weak topology on the simplex of probability measures on the compact space
([15]).
Remark 2. In the framework of solution of the finite-dimensional transportation problem, the optimal Lipschitz function
is nothing more than the Kantorovich–Gavurin potential from [6].
There is a huge number of difficult problems related to explicit calculation of the Kantorovich metric for a given compact space. For
, this is the classical Monge's problem on transportation of sand.
For
, there is a good answer: let
and
be two probability measures on
, and let
be the ordinary (Euclidean) metric; then
, i.e., the Kantorovich metric is just the
-metric for distribution functions. Apparently, there are no explicit formulas for
,
. Many papers are devoted to this problem; we will mention only the recent surveys [29, 30, 5].
However, it makes sense to mention an essential idea, which has appeared recently and which plays a very important role in modern applications to hydrodynamics, differential equations, and other areas (see [4,5,30] and references there); I mean the
-Kantorovich norms (see [2]). Namely, the original definition of the Kantorovich metric (and Kantorovich norm) resembles the definition of the
-norm; but we can also define an analog of the
-norm
where the infimum is taken, as before, over all transportation plans
for a pair of probability measures
, and the corresponding norm
for all
. Of course, the original Kantorovich metric (the case
) has more physical significance, but the case
is much more convenient from the technical and geometric point of view. The corresponding variational problem and Euler equation are simpler than in the case
, and the results of [2] show that for a certain geometric transportation problem, the Euler equation is the well-known Monge–Ampère equation (which a priori has nothing to do with the Monge–Kantorovich problem).
Let us mention another important special case, which is sometimes also called the MK-problem; we will call it the strong MK-problem.
Namely, with the above notation, it is formulated as follows: to find
where the infimum is taken over all measurable mappings
such that
.
The existence of minimum in (2) is a very subtle question. Of course,
, and the question of when the inequality becomes an equality is difficult and very important. In the last section, we will present a new approach to both problems.
Among a huge number of applications of the Kantorovich metric, I would like to mention only three examples, which are little known to specialists in applications of this metric, yet are very important in dynamical systems and functional analysis.
3 The iterated Kantorovich metric and the tower of measures
We will begin with the notion of tower of measures, which was defined in [20] and considered in more detail in [23], [39] Let
be an arbitrary compact metric space (say, the unit interval with the Euclidean metric). We can consider a new compact space
, the space of all probability Borel measures on
, and supply it with the Kantorovich metric. Thus we have defined a functor
from the category of metric compact spaces to itself:
,
; it is clear that
sends each homeomorphism of a compact space
to a compact space
to a homeomorphism of
to
.
Obviously,
can be isometrically embedded into
via the mapping
.
Let us iterate this procedure:
Set
and
and introduce the notation
for the mapping
.
We can consider the inductive limit of this sequence of metric spaces with isometric embeddings:
This inductive limit (a metric space) is called the infinite tower of measures; it plays a crucial role in the theory of filtrations of
-fields generated by random processes and its various applications.
On the other hand, for
there is a natural projection
where
is the barycenter of the measure
, which is well defined for measures on affine compact spaces (thus the projection is defined for
,
), and we have the sequence
Thus we obtain the projective limit
Since
, the inductive limit
is naturally embedded into the projective limit:
but, in contrast to the case of inductive limit, on the projective limit there is no natural metric.3
The main application of this tower of measures is as follows. Assume that we have a “metric triple”
, i.e., a measure space with a metric or semimetric, and a decreasing sequence of measurable partitions of this space (discrete filtration)
,
; here
is trivial and
.
First consider one partition
; for almost all points
of the quotient space with respect to this partition, there is a well-defined conditional measure on the element of
corresponding to
. We regard it as a measure on
; thus we have a mapping
, which sends almost every point
to a (conditional) measure on
. It is convenient to regard this mapping as a function from
to
.
Now define a metric (or semimetric) on
as follows: for almost all pairs of points
, define the distance between them as the Kantorovich distance between the corresponding conditional measures.
Thus we have defined a metric (or semimetric) on a subset of full measure in the quotient space
; it can also be regarded as a semimetric on the original space
.
Apply this process to the decreasing sequence of partitions
:
start from
, then define a metric on
, a mapping
, and a partition
; now we have a mapping from
to
, a new metric on
, and a map
.
Continuing this process, we obtain mappings
from
to the iterated spaces
, or to the inductive limit
.
One of the main results of the theory of decreasing sequences ([20], [23]) is the following theorem.
Theorem 2.
A decreasing homogeneous sequence of measurable partitions is standard (see [20, 23] for definitions) if and only if the sequence of measures
(in other words, the sequence of the distributions of the mappings
with respect to the measure
), regarded as a sequence of measures on the inductive limit
, tends to a
-measure.
A discussion of these subjects can be found in [23] and in forthcoming papers.
4 The Kantorovich metric in Ornstein's theory
In the early 70s, Donald Ornstein solved a long-standing problem in ergodic theory: he gave necessary and sufficient conditions on a discrete-time stationary random process under which the shift in the space of trajectories of this process is isomorphic to a Bernoulli shift; using this result, he proved that the Kolmogorov entropy is a complete invariant of Bernoulli shifts ([17]). We will formulate the main theorem of Ornstein's theory in order to illustrate the role of the Kantorovich metric, which was rediscovered by Ornstein (he called it the
-metric).
Assume that the state space
of a stationary process is finite and
is the stationary measure on
generated by this process. The question is formulated as follows: when there exists an isomorphism (in the measure-theoretic sense) of the Bernoulli space
with product measure and the space
that commutes with the shift. This is the well-known isomorphism problem in ergodic theory. It is clear that the criterion of existence of such an isomorphism must be expressed in terms of the rate of decrease of the correlation between the past and the future of the process. There are many known conditions of this type, which are sometimes called “mixing conditions.” Most of such conditions known in the theory of stationary processes are too strong (Kolmogorov's, Rozenblatt's, Ibragimov's conditions, etc.). It turned out that the right notion is related to the Kantorovich metric on the space of words with the Hamming metric – this was discovered by D. Ornstein. Our interpretation slightly differs from the original one, but is closer to the previous context (see [29]).
Let
,
, be a stationary random process with finite state space
and shift-invariant measure
on
. Consider the “past” of the process:
; the projection of
to
will be denoted by
. Fix a point
and consider the conditional distribution on the
-future given a fixed past
:
this is a measure on the
-future
defined for almost all points
; it is an element of
, thus we have a mapping
defined almost everywhere.
Consider the Hamming metric on
:
where
and
stands for the number of points in a set; and let
be the Kantorovich metric on the space
of measures on the
-future.
Theorem 3.
[17, 24] Consider a stationary process
,
, and the right shift in the space of realizations generated by this process. An invertible encoding of this shift into a Bernoulli shift (in other words, a measure-preserving isomorphism of the shift in the space of realizations of the process and a Bernoulli shift) exists if and only if
(the integral of the value of the Kantorovich metric for the pair of conditional measures corresponding to a pair of points from
with respect to the product measure
).
The literal meaning of the above condition is very transparent: it means that the conditional distribution on the future given a fixed past asymptotically does not depend on the past; roughly speaking, there is only one type of distribution on the future; but a more precise sense of these words essentially depends on the choice of a metric on the space of realizations of the process (we should take the Hamming metric) and a metric on the spaces of measures (here we should use the Kantorovich metric); in general, the conclusion of the theorem will be false if we replace the Kantorovich metric by some other one (for example, by the variation metric).
The last formulation also motivates the definition of the so-called secondary entropy of a stationary process (see [24]). Define
as the image of the measure
(see above) under the mapping
; this is a measure on
. In the case of Bernoulli automorphisms, by Ornstein's theorem, the measure
tends to a
-measure as
. But for a general Kolmogorov stationary process (K-automorphism), this is not the case. More precisely, if the automorphism is not a Bernoulli automorphism, then the limit exists, but is not a
-measure. Thus it is natural to introduce a characteristic of the limiting measure. Namely, we may consider the so-called
-entropy of the measure
. This notion also uses the Kantorovich metric. For an arbitrary Borel probability measure
on a metric space
, the
-entropy
(as a function of
) is defined as follows:
where the infimum is taken over all discrete measures
on
and
is the ordinary entropy of a discrete measure:
,
,
,
,
.
The asymptotic of
with respect to
is called the secondary entropy of the process. An open problem: what kind of asymptotic behavior can appear? Presumably, the secondary entropy is a metric invariant of K-automorphisms.
5 Application to the classification of metric spaces
Consider a Polish (=metric, complete, separable) space with a Borel probability measure. We call such a space a metric triple (another term is an
-space [7]). Two triples
and
are isomorphic if there exists a mapping
that is an isometry and preserves the measures:
and
.
We regard the metric as a measurable function of two variables:
(The theorem below is true for an arbitrary symmetric measurable function
, not necessarily a metric.) Let
be the product of infinitely many copies of the space
.
Define a mapping
from
to the set of symmetric matrices as follows:
, where
and
.
Let us denote the image of the measure
under the mapping
by
; the measure
on
will be called the matrix distribution of the function
.
In [25], we considered and classified general (nonsymmetric) measurable functions
of two variables on the space
up to mappings of the form
, where
and
are measure-preserving automorphisms of
. We also defined the notion of matrix distribution for this case; it is a complete invariant for so-called pure functions.
But now we need another classification. We also consider arbitrary measurable (nonsymmetric) functions
on the space
, where
is a Lebesgue space with continuous measure, but we classify them up to mappings of the form
, where
is an automorphism of
(in other words,
). Namely, define a mapping
where
and
; here
is the set of arbitrary (not necessarily symmetric) matrices.
The
-image of the measure
, which is a measure on
, is called the symmetric matrix distribution of the function
and denoted by
.
Theorem 4.
(Gromov [7], Vershik [25]) (1) Two metric triples
and
are isomorphic if and only if their matrix distributions coincide:
In other words, the matrix distribution of the metric is a complete invariant of a metric triple.
(2) (Vershik [25]). The symmetric matrix distribution
of a measurable function
of two variables is a complete metric invariant of the function regarded up to automorphisms of the form
, where
is an automorphism of
.
Now we apply this classification to MK-problems. Let
be a compact metric space with metric
; we want to “transport” a Borel probability measure
to another Borel probability measure
. Thus we have two metric triples:
and
. It is more convenient to reduce the problem to a more symmetric form and to have one metric triple. Let us consider only continuous measures; then we can choose a measure-preserving isomorphism
. Let
, so that
is a nonnegative measurable (in general, nonsymmetric) function of two variables – the “shifted metric.” Now we can consider only one measure
and the function
on the space
.
In terms of the shifted metric, the MK-problem can be formulated as follows: to find
where
runs over all Borel measures on the product
with both marginal measures equal to the measure
; thus
belongs to the set of bistochastic measures, or, in other words,
is an element of the semigroup of polymorphisms with invariant continuous measure
(see [22]) for definitions). Thus the MK-problem turns into a variational problem on the convex set of bistochastic measures (or on the semigroup of polymorphisms).
The strong MK-problem reads as follows: to find
where
runs over all
-preserving transformations of
. In this case, we have a variational problem on the group of measure-preserving transformations.
Now we can apply the above-defined symmetric matrix distribution
of the function
regarded as a measurable function (shifted metric) on the space
. Since
is a complete invariant of the triple
, all properties of the (ordinary and strong) MK-problem can be expressed in terms of
as a measure on the space of matrices
. But this means that we have a random matrix with distribution
, which we can use for analysis of the problem. Here we describe only one example of applying this approach.
Let
be a random matrix with distribution
. The new version of the MK-problem reads as follows. Choose a random matrix
, for each
consider the ordinary finite transportation problem, and define
where
is a bistochastic matrix (i.e.,
,
for all
) and
is the
-fragment of
(the random matrix constructed from the shifted metric as described above). Thus
is a random variable that depends on the random matrix
.
Theorem 5.
In the previous notation,
where
is the solution of the original MK-problem, i.e., the sequence of random variables
converges in measure
to the solution of the MK-problem.
A natural conjecture: for almost every choice of the matrix
with respect to the measure
, the same assertion is true:
which means that
converges to
with probability one with respect to the choice of the matrix
according to the measure
.
Note that we approximate the MK-problem with the simplest finite-dimensional problem of linear programming – the allocation problem. By the Birkhoff–von Neumann theorem, the solution of this problem is a permutation, i.e., an element of the symmetric group, or an extreme point of the convex set of bistochastic matrices (the so-called Hungarian polytope).
Nevertheless, the question of when the strong MK-problem has a solution and how it can be approximated by permutations is more involved.
The theorem and conjecture given above are typical for applications of our method to various problems with integral kernel: we obtain a probabilistic approximation of a functional or variational problem using a random choice of values of the function. We will return to this elsewhere.
Partially supported by the RFBR, project 02-01-00093, and the President of Russian Federation grant for support of leading scientific schools NSh-2251.2003.1. Translated by A. M. Vershik and N. V. Tsilevich. References
-
A. Barvinok, A Course in Convexity, Amer. Math. Soc., Providence, Rhode Island (2002).
-
Y. Brennier,”Extended Monge–Kantorovich theory”, Lecture Notes in Math., 1813, 91–122 (2003).
-
M. Émery, “Espaces probabilisés filtrés: de la théorie de Vershik au mouvement brownien, via les idées de Tsirelson,” Séminaire BOURBAKI, No. 882 (2000).
-
U. Frish, Turbulence. The Legacy of A. N. Kolmogorov, Cambridge Univ. Press, Cambridge (1995).
-
W. Gangbo and R. J. McCann, “The geometry of optimal transportation,” Acta Math., 177, No. 2, 113–161 (1966).
-
M. L. Gavurin and L. V. Kantorovich, “Application of mathematical methods to problems of analysis of freight flows,” in: Problems of Raising the Efficiency of Transport Performance [in Russian], Moscow–Leningrad (1949), pp. 110–138.
-
M. Gromov, Metric Structures for Riemannian and Non-Riemannian Spaces, Birkhäuser, Boston (1999).
-
L. V. Kantorovich, Mathematical Methods in the Organization and Planning of Production [in Russian], Leningrad (1939).
-
L. V. Kantorovich, “On an efficient method of solving some classes of extremal problems,” Dokl. Akad. Nauk SSSR, 28, No. 3, 212–215 (1940).
-
L. V. Kantorovich, “On the translocation of masses,” Dokl. Akad. Nauk SSSR, 37, Nos. 7–8, 227–229 (1942).
-
L. V. Kantorovich, “On a problem of Monge,” Uspekhi Mat. Nauk, 3, No. 2, 225–226 (1948).
-
L. V. Kantorovich, “Functional analysis and applied mathematics,” Uspekhi Mat. Nauk, 3, No. 6, 89–185 (1948).
-
L. V. Kantorovich, Economical Calculation of the Best Use of Resources [in Russian], Moscow (1960).
-
L. V. Kantorovich, “On new approaches to computational methods and processing of observations,” Sib. Mat. Zhurn., 3, No. 5, 701–709 (1962).
-
L. V. Kantorovich and G. Sh. Rubinshtein, “On a space of totally additive functions,” Vestn Lening. Univ., 13, No. 7, 52–59 (1958).
-
Leonid Vitalievich Kantorovich: Man and Scientist, vol. 1, Novosibirsk (2002).
-
D. Ornstein, Ergodic Theory, Randomness, and Dynamical Systems, Yale Univ. Press, New Haven–London (1974).
-
S. T. Rachev, Probability Metrics and the Stability of Stochastic Models, Wiley, Chichecter (1991).
-
A. M. Vershik, “Some remarks on infinite-dimensional problems of linear programming,” Uspekhi Mat. Nauk, 25, No. 5, 117–124 (1970).
-
A. M. Vershik, “Decreasing sequences of measurable partitions and their applications,” Sov. Math. Dokl., 11, No. 4, 1007–1011 (1970).
-
A. M. Vershik, “On D. Ornstein's papers, weak dependence conditions and classes of stationary measures,” Theory Probab. Appl., 21 (1977), 655–657.
-
A. M. Vershik, “Multivalued mappings with invariant measure (polymorphisms) and Markov operators,” J. Sov. Math., 23, 2243–2266 (1983).
-
A. M. Vershik, “Theory of decreasing sequences of measurable partitions,” St. Petersburg Math. J., 6, No. 4, 705–761 (1994).
-
A. M. Vershik, “Dynamic theory of growth in groups: entropy, boundaries, examples,” Russian Math. Surveys, 55, No. 4, 667–733 (2000).
-
A. M. Vershik, “Classification of measurable functions of several arguments, and invariantly distributed random matrices,” Funct. Anal. Appl., 36, No. 2, 93–105 (2002).
-
A. M. Vershik, “About L. V. Kantorovich and linear programming,” in: Leonid Vitalievich Kantorovich: Man and Scientist, vol. 1, Novosibirsk (2002), pp. 130–152.
-
A. Vershik, “Polymorphims, Markov processes, quasi-similarity of K-automorphisms,” to appear in Discrete Contin. Dyn. Syst.
-
A. M. Vershik and M. M. Rubinov, “General duality theorem in linear programming,” in: Mathematical Economics and Functional Analysis [in Russian], Nauka, Moscow (1974), pp. 35–55.
-
C. Villani, Topics in Optimal Transportation, Amer. Math. Soc., Providence, Rhode Island (2000).
-
Optmal Transportation and Applications. Springer Lecture Notes in Mathematics. Edit.L.A.Cafarelli, S.Salsa. v. 1813 (2003).
Mathematical Institute of Russian Ac.Sci. St.Petersburg branch, Fontanka 27, St.Petersburg, 191023, Russia. E-mail address : vershik@pdmi.ras.ru