, ,

The aim of this paper is to define a global and intrinsic notion of the principal symbol for the hypoelliptic calculus on Heisenberg manifolds, along with some applications. On the way, we complete the treatment of the Kohn Laplacian of [BG] and gives a complete treatment for the horizontal sublaplacian on a Heisenberg manifold acting on forms.

Recall that a Heisenberg manifold

(M, H)

consists of a manifold

M

together with a distinguished hyperplane bundle

H \subset T M

. This definition includes as main examples the Heisenberg group and its quotients by cocompact lattices, (codimension 1) foliations, CR and contact manifolds, as well as the confolations of [ET] . Moreover, in this context the natural operators such as Hörmander's sum of squares, the Kohn Laplacian, the horizontal sublaplacian or the contact Laplacian of [Ru] , while they may be hypoelliptic, are definitely not elliptic. Thus, the standard elliptic calculus cannot be used efficiently to study these operators.

The relevant substitute to the elliptic calculus is the Heisenberg calculus of Beals-Greiner [BG] and Taylor [Tay] , which extends seminal works of Boutet de Monvel [Bo1] , Dynin ([Dy1] , [Dy2] ) and Folland-Stein [FS] (see also [BGH] , [CGGP] , [EMM] , [Gri] , [H¨3] ). The idea in the Heisenberg calculus,which goes back to Elias Stein, is to build a pseudodifferential calculus on Heisenberg manifoldsmodelled on that of left-invariant pseudodifferential operators on nilpotent groups. This stems fromthe fact that the relevant tangent structure for a Heisenberg manifold

(M, H)

is that of a bundle

G M

of two-step nilpotent Lie groups (see [BG] , [Be] , [EMM] , [FS] , [Gro] , [Po2] , [Ro2] ). In the original monographs [BG] and [Tay] the principal symbol is only defined in local coordinates,so the definition a priori depends on the choice of these coordinates. In the special case of a contactmanifold, an intrinsic definition have been given in the unpublished book [EMM] , as a section overa bundle of jets of vector fields representing the tangent group bundle of the contact manifold.This approach is similar to that of Melrose [Me2] in the setting of the

b

-calculus for manifolds withboundary. In this paper we give an intrinsic definition of the principal symbol, valid for an arbitraryHeisenberg manifold, in terms of the description of the tangent Lie group bundle in [Po2] . As aconsequence, we can reformulate in a global fashion previously known criterions for existence ofparametrices in the Heisenberg calculus. In particular, we can define a Rockland condition foroperators in the Heisenberg calculus in a fully intrinsic way and show that this condition allowsto determine the existence of a parametrix when the Levi form of the Heisenberg manifold hasconstant rank. More importantly, since our approach of the principal symbol connects nicely with the constructionof the tangent groupoid of a Heisenberg manifold in [Po2] , this presumably allows us to make useof global

K

-theoretic arguments in the Heisenberg setting, as those involved in the proof of the(full) Atiyah-Singer index theorem ([AS1] , [AS2] ) and the regularity of the eta invariant for generalselfadjoint elliptic

Ψ

DO's ([APS] , [Gi] ) or, equivalently, the vanishing of the noncommutative residueof a

Ψ

DO projection (see [Wo1] , [BL] , [Po1] ). More precisely, this paper can also be seen as a steptowards extending the aforementioned results to the Heisenberg setting. On the other hand, for strictly pseudoconvex CR manifolds this should allows us to recover andto extend in a rather simple way a recent result of Hirachi [Hi] on the invariance of the integral ofthe logarithmic singularity of the Szegö kernel (see also [Bo2] ), in connection with the program of Fefferman [Fe] for the Szegö and the Bergman kernels on strictly pseudoconvex complex domains.

1.1 Intrinsic notion of principal symbol for the Heisenberg calculus

Given a Heisenberg a manifold

(M, H)

and a vector bundle

ℰ

we let

g^{*} M

denote the linear dual of the Lie algebra bundle associated to the tangent Lie group bundle. For

m \in C

we let

Ψ^{m} H (M, ℰ)

denote the class of

Ψ H

DO's of order

m

acting on section of

ℰ

and, letting

π : g^{*} M \to M

be the canonical projection, we define

S_{m} (M, ℰ)

as the space of sections

p_{m} \in C^{\infty} (g^{*} M \ 0, End π_{*} ℰ)

such that for any

λ > 0

we have

\begin{matrix} p_{m} (λ^{2} ξ_{0}, λ ξ^{'}) = λ^{m} p_{m} (ξ_{0}, ξ^{'}), ξ_{0} \in (T M / H)^{*}, ξ^{'} \in H^{*} . \end{matrix}

(1.1)

The principal symbol of an operator

P \in Ψ^{m} H (M, ℰ)

is obtained as an element of

S_{m} (g^{*} M, ℰ)

as follows. As shown in [Po2] the tangent Lie group bundle

G M

of a Heisenberg manifold

(M, H)

can be described as the bundle

(T M / H) \oplus H

endowed with a law group encoded by the Levi form

ℒ : H \times H \to T M / H

such that, for sections

X

and

Y

H

, we have

\begin{matrix} ℒ (X, Y) = [X, Y] m o d H . \end{matrix}

(1.2)

It is also shown in [Po2] that at a point

x \in M

this approach to

G M

is equivalent to the extrinsic one in [BG] in terms of the Lie group of a nilpotent Lie algebra of jets of vector fields at point of a local chart. This is made by means of a special kind of privileged coordinates at

x

, called Heisenberg coordinates.

As a byproduct of the equivalence between the two approaches, we obtain that in Heisenberg coordinates a Heisenberg diffeomorphism

φ : (M, H) \to (M^{'}, H^{'})

is well approximated by the induced isomorphism

φ_{H}^{'} : G M \to G M^{'}

between the tangent Lie group bundles (see [Po2] ). This allows us to carry out the proof of the invariance by Heisenberg diffeomorphisms of the Heisenberg calculus in Heisenberg coordinates, rather than in privileged coordinates as in [BG] (see Section 8 ).

As a consequence we can prove:

Theorem 1.1. Let

P : C_{c}^{\infty} (M, ℰ) \to C^{\infty} (M, ℰ)

be a

Ψ H

DO of order

m

. Then there exists a unique symbol a unique symbol

σ_{m} (P) \in S_{m} (g^{*} M, ℰ)

such that, for any

a \in M

, the symbol

σ_{m} (P) (a, .)

agrees in trivializing Heisenberg coordinates centered at

a

with the principal symbol of

P

x = 0

We call the symbol

σ_{m} (P) (x, ξ)

the principal symbol of

P

. In order to distinguish it from the other definition of the principal symbol in local coordinates, we will sometimes refer to it as the global principal symbol, while the other principal symbol will be called the local principal symbol.

Let us now describe the main properties of the global principal symbol. First, we have:

Proposition 1.2. For every

m \in C

the principal symbol map

σ_{m} : Ψ^{m} H (M, ℰ) \to S_{m} (g^{*} M, ℰ)

gives rise to a linear isomorphism

Ψ^{m} H (M, ℰ) / Ψ^{m - 1} H (M, ℰ) \tilde{⟶} S_{m} (g^{*} M, ℰ)

P \in Ψ^{m} H (M, ℰ)

has principal symbol

σ_{m} (P)

then for any

a \in M

we define the model operator

P^{a}

P

a

as the operator as the left-invariant

Ψ H

DO on

G_{a} M

with symbol

σ_{m} (a, .)

, that is, the left-convolution operator with the inverse Fourier transform of

σ_{m} (a, .)

(see Definition 4.7 ).

On the other hand, for any

a \in M

the product law on

G_{a} M

defines a product for symbols,

\begin{matrix} *^{a} : S_{m_{1}} (g_{a}^{*} M) \times S_{m_{2}} (g_{a}^{*} M) ⟶ S_{m_{1} + m_{2}} (g_{a}^{*} M) . \end{matrix}

(1.3)

This product depends smoothly enough on

a

to give rise to a product

\begin{matrix} * : S_{m_{1}} (g^{*} M, ℰ) \times S_{m_{2}} (g^{*} M, ℰ) ⟶ S_{m_{1} + m_{2}} (g^{*} M, ℰ), \end{matrix}

(1.4)

such that for

p_{j} \in S_{m_{j}} (g^{*} M, ℰ)

j = 1, 2

, we have

\begin{matrix} p_{m_{1}} * p_{m_{2}} (a, ξ) = [p_{m_{1}} (a, .) *^{a} p_{m_{2}} (a, .)] (ξ) \forall (a, ξ) \in g^{*} M \ 0 . \end{matrix}

(1.5)

This comes from the fact that in local coordinates the above product is nicely related to the product of local homogeneous symbols of [BG] . As a consequence we get:

Proposition 1.3. For

j = 1, 2

let

P_{j} \in Ψ^{m_{j}} H (M, ℰ)

and suppose that

P_{1}

P_{2}

is properly supported.

1) We have

σ_{m_{1} + m_{2}} (P_{1} P_{2}) = σ_{m_{1}} (P) * σ_{m_{2}} (P)

2) At every point

a \in M

the model operator of

P_{1} P_{2}

(P_{1} P_{2})^{a} = P_{1}^{a} P_{2}^{a}

Next, it is shown in [BG] that the transpose of a

Ψ H

DO is again a

Ψ H

DO. Thanks to the results of [Po2] , we can prove a version of this result in Heisenberg coordinates (see Section 9 ). As a consequence, we identify the principal symbol of transpose, for we get:

Proposition 1.4. Let

P \in Ψ^{m} H (M, ℰ)

have principal symbol

σ_{m} (P)

. Then: 1) The principal symbol of the transpose

P^{t}

σ_{m} (P^{t}) (x, ξ) = σ_{m} (x, - ξ)^{t}

; 2) If

P^{a}

is the model operator of

P

a

, then the model operator of

P^{t}

a

is the transpose operator

(P^{a})^{t} : S_{0} (G_{x} M, ℰ_{x}^{*}) \to S_{0} (G_{x} M, ℰ_{x}^{*})

Assume now that

M

is endowed with a positive density and

ℰ

with a Hermitian metric, and let

L^{2} (M, ℰ)

be the associated

L^{2}

-Hilbert space. Then we have:

Proposition 1.5. Let

P \in Ψ^{m} H (M, ℰ)

have principal symbol

σ_{m} (P)

. Then: 1) The principal symbol of the adjoint

P^{*}

σ_{\bar{m}} (P^{*}) (x, ξ) = σ_{m} (P) (x, ξ)^{*}

2) If

P^{a}

denotes the model operator of

P

a \in M

then the model operator of

P^{*}

a

is the adjoint

(P^{a})^{*}

P^{a}

1.2 Rockland condition and hypoellipticity

It is shown in [BG] that, in local coordinates, the invertibility of the local principal symbol is equivalent to the existence of a

Ψ H

DO parametrix. Thanks to Proposition 1.3 and the relationship in local coordinates between the local and global principal symbols, we can reformulate this result in a global fashion. More precisely, we have:

Proposition 1.6. Let

P : C_{c}^{\infty} (M, ℰ) \to C^{\infty} (M, ℰ)

be a

Ψ H

DO of order

m

. Then the following are equivalent:

1) The principal symbol

σ_{m} (P)

P

is invertible with respect to the convolution product for homogeneous symbols; 2) The operator

P

admits a parametrix

Q

Ψ^{- m} H (M, ℰ)

Furthermore, if 1) and 2) hold then

P

is hypoelliptic with gain of

\frac{k}{2}

-derivatives.

In fact, it may be difficult to determine whether the principal symbol of a

Ψ H

DO is invertible because in general the convolution product for symbols is not the pointwise product of symbols. In particular, in the Heisenberg setting, the product of principal symbols is neither commutative, nor microlocal.

Nevertheless, to determine the invertibility of the principal symbol we can make use of a general representation theoretic criterion, the Rockland condition: a

Ψ H

DO-operator

P

satisfies the Rockland condition at a point

a \in M

when its model operator

P^{a}

satisfies the Rockland condition on

G_{a} M

, i.e., for every nontrivial irreducible unitary representation

π

G_{a} M

some (unbounded) operator

π_{P^{a}}

acting on the representation space of

π

is injective on the space of smooth vectors of

π

It is well known that a left-invariant homogeneous

Ψ

DO on a nilpotent group is hypoelliptic if, and only if, it satisfies the Rockland condition, and it further admits an inverse if, and only if, together with its transpose it satisfies the Rockland condition (e.g. [HN1] , [HN2] , [CGGP] ). Thus if

P

is a

Ψ H

DO such that

P

and

P^{t}

satisfies the Rockland condition at every point then, for any

x \in M

, then the model operator of

P^{x}

is invertible. However, whether this inverse depends smoothly enough with respect to

x

to yield an inverse for the principal symbol is a more delicate issue.

We show here that this occurs when the Levi form ( 1.2 ) has constant rank, say

2 n

. In this case the tangent Lie group is a fiber bundle with typical fiber

H^{2 n + 1} \times R^{d - 2 n}

. This allows us to make use of results of Christ et al. [CGGP] about families of

Ψ

DO's on a fixed nilpotent graded group to get:

Proposition 1.7. Let

P

be a

Ψ H

DO of order

m

ℜ m \geq 0

, and suppose that the Levi form ( 1.2 ) has constant rank. Then the following are equivalent:

(i)

P

and

P^{t}

satisfy the Rockland condition at every point of

M

; (ii)

P

and

P^{*}

satisfy the Rockland condition at every point of

M

; (iii) The principal symbol of

P

is invertible.

In particular, if (i) and (ii) holds then

P

admits a

Ψ H

DO parametrix and is hypoelliptic with loss of

\frac{1}{2} ℜ m

derivatives.

In particular, when

P

is selfadfoint and the Levi form has constant rank the Rockland condition for

P

is equivalent to the invertibility of the principal symbol of

P

Finally, if

2 n

denotes the rank of the Levi form at

a

then we have

G_{a} M ≃ H^{2 n + 1} \times R^{d - 2 n}

Therefore, the irreducible unitary representations of

G_{a} M

are two kinds, one dimensional representations on

C

and infinite dimensional representations on

L^{2} (R^{n})

. In the former case the Rockland condition corresponds to the invertibility of the principal symbol along

H^{*}

, while in the latter case it is enough to look at the representations coming from that of

H^{2 n + 1}

with Planck constants

\pm 1

(see Section 5 ).

1.3 Hypoellipticity criterions for sublaplacians

It is interesting to look at the previous results in the case of sublaplacians, as this covers several important examples such as Hörmander's sum of squares, the Kohn Laplacian or the horizontal sublaplacian.

Here by sublaplacian we mean a differential operator

Δ : C^{\infty} (M, ℰ) \to C^{\infty} (M, ℰ)

such that near every point

a \in M

we can write

Δ

in the form,

\begin{matrix} Δ = - (X_{1}^{2} + \dots + X_{d}^{2}) - i μ (x) X_{0} + O_{H} (1), \end{matrix}

(1.6)

where

X_{0}, X_{1}, \dots, X_{d}

is a

H

-frame of

T M

, so that

X_{1}, \dots, X_{d}

span

H

, the term

μ (x)

is a smooth section of

End ℰ

and the notation

O_{H} (1)

stands for a differential operator of Heisenberg order

\leq 1

In this case the Rockland condition and the invertibility of the reduces to the following. Let

L (x) = (L_{j k} (x))

be the matrix of

ℒ

with respect to the

H

-frame

X_{0}, X_{1}, \dots, X_{d}

, so that for

j, k = 1, \dots, d

we have

\begin{matrix} ℒ (X_{j}, X_{k}) = [X_{j}, X_{k}] = L_{j k} (x) X_{0} m o d H . \end{matrix}

(1.7)

Let

2 n

be the rank of

ℒ_{a}

and

L (a)

, let

λ_{1}, \dots, λ_{d}

denoting the eigenvalues of

L (a)

and consider the condition,

\begin{matrix} Sp μ (a) \cap Λ_{a} = \emptyset, \end{matrix}

(1.8)

where the singular set

Λ_{a}

is defined as follows,

\begin{matrix} \begin{matrix} (1.9) & Λ_{a} = (- \infty, - \frac{1}{2} Trace | L (a) |] \cup [\frac{1}{2} Trace | L (a) |, \infty) if 2 n < d, Λ_{a} = {\pm (\frac{1}{2} Trace | L (a) | + 2 α_{j} | λ_{j} |); α_{j} \in N^{d}} if 2 n = d . \end{matrix} \end{matrix}

(1.10)

As is turns out this condition makes sense independently of the choice of the

H

-frame and is the relevant condition to look at in the case of a sublaplacian. More precisely, we have:

Proposition 1.8. 1) The condition ( 6.23 ) makes sense intrinsically for any

a \in M

2) At every point

a \in M

the Rockland conditions for

Δ

and

Δ^{t}

are equivalent to ( 1.8 ).

3) The principal symbol of

Δ

is invertible if, and only if, the condition ( 1.8 ) holds at every point of

M

. Moreover, when the latter occurs

Δ

admits a parametrix in

Ψ^{- 2} H (M, ℰ)

and is hypoelliptic with loss of one derivative.

The above result is proved in [BG] in the case of scalar sublaplacians, for which the coefficient

μ (x)

in ( 1.6 ) is a complex-valued function, but the general case is not dealt with in [BG] . However, it is necessary to extend the result to sublaplacians acting on sections of vector bundles in order to deal with the Kohn Laplacian and the horizontal sublaplacian acting on forms. In particular, Proposition 1.8 allows us to complete the treatment of the Kohn Laplacian in [BG] (see below).

1.4 Main examples of hypoelliptic operators on Heisenberg manifolds

We devote a section where we explain how the Heisenberg calculus, including the results of this paper, provides us with a unified framework to deal with the hypoellipticity of the main geometric operators on a Heisenberg manifold and to recover well-known results proved using various different approaches.

(a) Hörmander's sum of squares

A Hörmander's sum of squares is an operator of the form

Δ = - (X_{1}^{2} + \dots + X_{m}^{2})

where

X_{1}, \dots, X_{m}

are vector fields on

M

. When

X_{1}, \dots, X_{m}

span

H

we get a sublaplacian and Proposition 1.8 allows us to recover, in this special case, the celebrated result of Hörmander [H¨2] about thehypoellipticity of sum of squares under the bracket condition.

(b) Kohn Laplacian

The Kohn Laplacian is the Laplacian associated to the tangential Cauchy-Riemann complex, or

\bar{\partial} b

-complex, on a CR manifold ([KR] , [Koh1] ). It was shown by Kohn [Koh1] that under the condition

Y (q)

the Kohn Laplacian acting on

(p, q)

-forms is hypoelliptic with loss of one derivative.

It was proved by Beals-Greiner [BG] that for the Kohn Laplacian acting on

(p, q)

-forms the condition ( 1.8 ) reduces to the condition

Y (q)

, so we may apply Proposition 1.8 to recover Kohn's result. This allows us to complete the treatment of the Kohn in [BG] , because the initial argument there is not quite complete (see Remark 7.3 ).

The horizontal sublaplacian

Δ_{b}

on a Heisenberg manifold

(M^{d + 1}, H)

can be seen as a horizontal Laplacian acting on the horizontal forms, that is on the sections of

Λ_{C}^{*} H^{*}

. This operator was first introduced by Tanaka [Ta] for strictly pseudoconvex CR manifolds, but versions of this operator acting on functions were independently defined by Greenleaf [Gr] and Lee [Le] .

While

Δ_{b}

acting on functions can be seen as a sum of squares up to lower order terms, it seems that little has been done concerning the hypoellipticity of

Δ_{b}

acting on forms on a general Heisenberg manifolds, except in the contact case (see [Ta] , [Ru] ).

We show here that the invertibility of the principal symbol of

Δ_{b}

acting on the sections of

Λ_{C}^{k} H^{*}

reduces to a condition involving

k

and the Levi form of

(M, H)

only. More precisely, for

a \in M

let

2 n

be the rank of the Levi form

ℒ

a

. We say that the condition

X (k)

is satisfied at

a

when

\begin{matrix} k \notin {n, n + 1, \dots, d - n} . \end{matrix}

(1.11)

We then prove that this condition is the condition ( 1.8 ) for

Δ_{b}

acting on

Λ_{C}^{k} H^{*}

. Thus, using Proposition 1.8 we get:

Proposition 1.9. Let

Δ_{b} : C^{\infty} (M, Λ_{C}^{k} H^{*}) \to C^{\infty} (M, Λ_{C}^{k} H^{*})

be the horizontal sublaplacian acting on horizontal forms of degree

k

1) At a point

x \in M

the Rockland condition for

Δ_{b}

is equivalent to the condition

X (k)

2) The principal symbol of

Δ_{b}

is invertible if, and only if, the condition

X (k)

is satisfied at every point.

(d) Contact Laplacian

Given a contact manifold

(M^{2 n + 1}, θ)

the contact Laplacian is associated to the contact complex defined by Rumin [Ru] . Unlike the previous examples this not a sublaplacian and it is even of order 4 on contact forms of middle degrees.

It has been shown by Rumin [Ru] that the contact Laplacian satisfies the Rockland condition on forms of any degree. Rumin then used results of Helffer-Nourrigat [HN3] to deduce that the contact Laplacian was hypoelliptic maximal. Alternatively, we may use Proposition 1.7 to deduce that on contact forms of any degree the contact Laplacian has an invertible principal symbol, hence admits a parametrix in the Heisenberg calculus and is hypoelliptic.

1.5 Organization of the paper

The paper is organized as follows. In Section 2 we recall the main definitions and examples concerning Heisenberg manifolds and their tangent Lie group bundles. In Section 3 we give a detailed overview of the Heisenberg calculus of [BG] and [Tay] , following closely the exposition of [BG] .

The section 4 is devoted to the definitions and the main properties of the principal symbols and model operators of

Ψ H

DO's. In Section 5 we study the relationships between invertibility of the principal symbol, Rockland condition and hypoellipticity and, in particular, we prove Theorem 1.7 .

In Section 6 we deal with sublaplacians and in particular extend the results of [BG] to sublaplacians acting on sections of vector bundles. In Section 7 we deal with the invertibility of the principal symbols of main geometric operators on Heisenberg manifolds: Hörmander's sum of squares, Kohn Laplacian, horizontal sublaplacian and contact Laplacian. In particular, we prove Proposition 1.9 .

The last two sections are devoted to the proofs of Proposition 3.18 and Proposition 3.21 . These are versions in Heisenberg coordinates of the invariance of the Heisenberg calculus by Heisenberg diffeomorphisms and transposition. The latter were proved in [BG] in a less precise form, but in this paper we need the precise version in Heisenberg coordinates in order to show that the definition of the principal symbol makes sense intrinsically and to determine the principal symbols of the transpose and the adjoint of a

Ψ H

DO.

Acknowledgements. The author is grateful to the hospitality of the Mathematics Departments of Princeton University and Harvard University where most part of this paper was written. The research of the author was partially supported by the NSF grant DMS 0409005.

2 Heisenberg manifolds and their tangent Lie group bundles

In this section we recall the main facts about Heisenberg manifolds and their tangent Lie group bundle. The exposition here follows closely that of [Po2] .

Definition 2.1. 1) A Heisenberg manifold is a smooth manifold

M

equipped with a distinguished hyperplane bundle

H \subset T M

2) A Heisenberg diffeomorphism

φ

from a Heisenberg manifold

(M, H)

onto another Heisenberg manifold

(M, H^{'})

is a diffeomorphism

φ : M \to M^{'}

such that

φ_{*} H = H^{'}

Definition 2.2. Let

(M^{d + 1}, H)

be a Heisenberg manifold. Then: 1) A (local)

H

-frame for

T M

is a (local) frame

X_{0}, X_{1}, \dots, X_{d}

T M

so that

X_{1}, \dots, X_{d}

span

H

2) A local Heisenberg chart is a local chart together with a local

H

-frame of

T M

over its domain.

The main examples of Heisenberg manifolds are the following.

(a) Heisenberg group. The

(2 n + 1)

-dimensional Heisenberg group

H^{2 n + 1}

R^{2 n + 1} = R \times R^{2 n}

equipped with the group law,

\begin{matrix} x . y = (x_{0} + y_{0} + \sum_{1 \leq j \leq n} (x_{n + j} y_{j} - x_{j} y_{n + j}), x_{1} + y_{1}, \dots, x_{2 n} + y_{2 n}) . \end{matrix}

(2.1)

A left-invariant basis for its Lie algebra

h^{2 n + 1}

is provided by the vector-fields,

\begin{matrix} X_{0} = \frac{\partial}{\partial x_{0}}, X_{j} = \frac{\partial}{\partial x_{j}} + x_{n + j} \frac{\partial}{\partial x_{0}}, X_{n + j} = \frac{\partial}{\partial x_{n + j}} - x_{j} \frac{\partial}{\partial x_{0}}, 1 \leq j \leq n, \end{matrix}

(2.2)

which for

j, k = 1, \dots, n

and

k \neq j

satisfy the Heisenberg relations,

\begin{matrix} [X_{j}, X_{n + k}] = - 2 δ_{j k} X_{0}, [X_{0}, X_{j}] = [X_{j}, X_{k}] = [X_{n + j}, X_{n + k}] = 0 . \end{matrix}

(2.3)

In particular, the subbundle spanned by the vector fields

X_{1}, \dots, X_{2 n}

defines a left-invariant Heisenberg structure on

H^{2 n + 1}

(b) Foliations. A (smooth) foliation is a manifold

M

together with a subbundle

ℱ \subset T M

which is integrable in the Froebenius' sense, i.e., we have

[ℱ, ℱ] \subset ℱ

. Thus, any codimension 1 foliation is a Heisenberg manifold.

M^{2 n + 1}

is given by a global non-vanishing

1

-form

θ

M

such that

d θ

is non-degenerate on

H = ker θ

. In particular,

(M, H)

is a Heisenberg manifold. In fact, by Darboux's theorem any contact manifold

(M^{2 n + 1}, θ)

is locally contact-diffeomorphic to the Heisenberg group

H^{2 n + 1}

equipped with its standard contact form

θ^{0} = d x_{0} + \sum_{j = 1}^{n} (x_{j} d x_{n + j} - x_{n + j} d x_{j})

(d) Confoliations. According to Elyashberg-Thurston [ET] a confoliation structure on an oriented manifold

M^{2 n + 1}

is given by a global non-vanishing

1

-form

θ

M

such that

(d θ)^{n} \land θ \geq 0

. In particular, when

(d θ)^{n} \land θ = 0

(resp.

(d θ)^{n} \land θ > 0

) we are in presence of a foliation (resp. a contact structure). In any case the hyperplane bundle

H = ker θ

defines a Heisenberg structure on

M

(e) CR manifolds. A CR structure on an orientable manifold

M^{2 n + 1}

is given by a rank

n

complex subbundle

T_{1, 0} \subset T_{C} M

which is integrable in Froebenius' sense and such that

T_{1, 0} \cap T_{0, 1} = {0}

, where

T_{0, 1} = \bar{T_{1, 0}}

. Equivalently, the subbundle

H = ℜ (T_{1, 0} \otimes T_{0, 1})

has the structure of a complex bundle of (real) dimension

2 n

. In particular, the pair

(M, H)

forms a Heisenberg manifold.

Moreover, since

M

is orientable and

H

is orientable by means of its complex structure, the normal bundle

T M / H

is orientable, hence admits a global nonvanishing section

T

. Let

θ

be the global section of

T^{*} M / H^{*}

such that

θ (T) = 1

and

θ

annihilates

H

. Then Kohn's Levi form is the form

L_{θ}

T_{1, 0}

such that, for sections

Z

and

W

T_{1, 0}

, we have

\begin{matrix} L_{θ} (Z, W) = - i d θ (Z, \bar{W}) = i θ ([Z, \bar{W}]) . \end{matrix}

(2.4)

We say that

M

is strictly pseudoconcex (resp. nondegenerate,

κ

-strictly pseudoconvex) when for some choice of

θ

the Levi form

L_{θ}

is everywhere positive definite (resp. is everywhere non-degenerate, has everywhere signature

(n - κ, κ, 0)

). In particular, when

M

is nondegenerate the 1-form

θ

is a contact form on

M

The main example of a CR manifold is that of the (smooth) boundary

M = \partial D

of a complex domain

D \subset C^{n}

. In particular, when

D

is strongly pseudoconvex (or strongly pseudoconcave)

M

is strictly pseudoconvex.

2.1 The tangent Lie group bundle

A simple description of the tangent Lie group bundle of a Heisenberg manifold

(M^{d + 1}, H)

can be given as follows.

Lemma 2.3 ([Po2] ). The Lie bracket of vector fields induces 2-form,

\begin{matrix} ℒ : H \times H ⟶ T M / H, \end{matrix}

(2.5)

such that, for any

a \in M

and any sections

X

and

Y

H

near

a

, we have

\begin{matrix} ℒ_{a} (X (a), Y (a)) = [X, Y] (a) m o d H_{a} . \end{matrix}

(2.6)

Definition 2.4. The

2

-form

ℒ

is called the Levi form of

(M, H)

The Levi form

ℒ

allows us to define a bundle

g M

of graded Lie algebras by endowing

(T M / H) \oplus H

with the smooth fields of Lie Brackets and gradings such that

\begin{matrix} [X_{0} + X^{'}, Y_{0} + Y^{'}]_{a} = ℒ_{a} (X^{'}, Y^{'}) and t . (X_{0} + X^{'}) = t^{2} X_{0} + t X^{'}, t \in R, \end{matrix}

(2.7)

for

a \in M

and

X_{0}

Y_{0}

T_{a} M / H_{a}

and

X^{'}

Y^{'}

H_{a}

Definition 2.5. The bundle

g M

is called the tangent Lie algebra bundle of

M

As we can easily check

g M

is a bundle of

2

-step nilpotent Lie algebras which contains the normal bundle

T M / H

in its center. Therefore, its associated graded Lie group bundle

G M

can be described as follows. As a bundle

G M

(T M / H) \oplus H

and the exponential map is the identity. In particular, the grading of

G M

is as in ( 2.7 ). Moreover, since

g M

is 2-step nilpotent the Campbell-Hausdorff formula gives

\begin{matrix} (exp X) (exp Y) = exp (X + Y + \frac{1}{2} [X, Y]), X, Y sections of g M . \end{matrix}

(2.8)

From this we deduce that the product on

G M

is such that

\begin{matrix} (X_{0} + X^{'}) . (Y_{0} + X^{'}) = X_{0} + Y_{0} + \frac{1}{2} ℒ (X^{'}, Y^{'}) + X^{'} + Y^{'}, \end{matrix}

(2.9)

for sections

X_{0}

Y_{0}

T M / H

and sections

X^{'}

Y^{'}

H

Definition 2.6. The bundle

G M

is called the tangent Lie group bundle of

M

In fact, the fibers of

G M

are classified by the Levi form

ℒ

as follows.

Proposition 2.7 ([Po2] ). 1) Let

a \in M

. Then

ℒ_{a}

has rank

2 n

if, and only if, as a graded Lie group

G_{a} M

is isomorphic to

H^{2 n + 1} \times R^{d - 2 n}

2) The Levi form

ℒ

has constant rank

2 n

if, and only if,

G M

is a fiber bundle with typical fiber

H^{2 n + 1} \times R^{d - 2 n}

Now, let

φ : (M, H) \to (M^{'}, H^{'})

be a Heisenberg diffeomorphism from

(M, H)

onto another Heisenberg manifold

(M^{'}, H^{'})

. Since

φ_{*} H = H^{'}

we see that

φ^{'}

induces a smooth vector bundle isomorphism

\bar{φ} : T M / H \to T M^{'} / H^{'}

Definition 2.8. We let

φ_{H}^{'} : (T M / H) \oplus H \to (T M^{'} / H^{'}) \oplus H^{'}

denote the vector bundle isomorphism such that

\begin{matrix} φ_{H}^{'} (a) (X_{0} + X^{'}) = {\bar{φ}}^{'} (a) X_{0} + φ^{'} (a) X^{'}, \end{matrix}

(2.10)

for any

a \in M

and any

X_{0} \in T_{a} / H_{a}

and

X^{'} \in H_{a}

Proposition 2.9 ([Po2] ). The vector bundle isomorphism

φ_{H}^{'}

is an isomorphism of graded Lie group bundles from

G M

onto

G M^{'}

. In particular, the Lie group bundle isomorphism class of

G M

depends only on the Heisenberg diffeomorphism class of

(M, H)

2.2 Heisenberg coordinates and nilpotent approximation of vector fields

It is interesting to relate the intrinsic description of

G M

above with the more extrinsic description of [BG] (see also [Be] , [EMM] , [FS] , [Gro] , [Ro2] ) in terms of the Lie group associated to a nilpotent Lie algebra of model vector fields.

First, let

a \in M

and let us describe

g_{a} M

as the graded Lie algebra of left-invariant vector fields on

G_{a} M

by identifying any

X \in g_{a} M

with the left-invariant vector fields

L_{X}

G_{a} M

given by

\begin{matrix} L_{X} f (x) = \frac{d}{d t} f [(t exp X) . x]_{|_{t = 0}} = \frac{d}{d t} f [(t X) . x]_{|_{t = 0}}, f \in C^{\infty} (G_{a} M) . \end{matrix}

(2.11)

This allows us to associate to any vector fields

X

near

a

a unique left-invariant vector fields

X^{a}

G_{a} M

such that

\begin{matrix} X^{a} = {\begin{matrix} L_{X_{0} (a)} & if X (a) \notin H_{a}, \\ L_{X (a)} & otherwise, \end{matrix} \end{matrix}

(2.12)

where

X_{0} (a)

denotes the class of

X (a)

modulo

H_{a}

Definition 2.10. The left-invariant vector fields

X^{a}

is called the model vector fields of

X

a

Let us look at the above construction in terms of a

H

-frame

X_{0}, \dots, X_{d}

near

a

, i.e. of a local trivialization of the vector bundle

(T M / H) \oplus H

. For

j, k = 1, \dots, d

we let

\begin{matrix} ℒ (X_{j}, X_{k}) = [X_{j}, X_{k}] X_{0} = L_{j k} X_{0} m o d H . \end{matrix}

(2.13)

With respect to the coordinate system

(x_{0}, \dots, x_{d}) \to x_{0} X_{0} (a) + \dots + x_{d} X_{d} (a)

we can write the product law of

G_{a} M

\begin{matrix} x . y = (x_{0} + \frac{1}{2} \sum_{j, k = 1}^{d} L_{j k} x_{j} x_{k}, x_{1}, \dots, x_{d}) . \end{matrix}

(2.14)

Then the vector fields

X_{j}^{a}

j = 0, \dots, d

, in ( 2.12 ) are the left-invariant vector fields corresponding to the vectors

e_{j}

j = 0, \dots, d

, of the canonical basis of

R d + 1

, i.e., we have

\begin{matrix} X_{0}^{a} = \frac{\partial}{\partial x_{0}} and X_{j}^{a} = \frac{\partial}{\partial x_{j}} - \frac{1}{2} \sum_{k = 1}^{d} L_{j k} x_{k} \frac{\partial}{\partial x_{0}}, 1 \leq j \leq d . \end{matrix}

(2.15)

In particular, for

j, k = 1, \dots, d

we have the relations,

\begin{matrix} [X_{j}^{a}, X_{k}^{a}] = L_{j k} (a) X_{0}^{a}, [X_{j}^{a}, X_{0}^{a}] = 0 . \end{matrix}

(2.16)

Now, let

κ : dom κ \to U

be a Heisenberg chart near

a = κ^{- 1} (u)

and let

X_{0}, \dots, X_{d}

be the associated

H

-frame of

T U

. Then there exists a unique affine coordinate change

x \to ψ_{u} (x)

such that

ψ_{u} (u) = 0

and

ψ_{u *} X_{j} (0) = \frac{\partial}{\partial x_{j}}

for

j = 0, 1, \dots, d

. Indeed, if for

j = 1, \dots, d

we set

X_{j} (x) = \sum_{k = 0}^{d} B_{j k} (x) \frac{\partial}{\partial x_{k}}

then we have

\begin{matrix} ψ_{u} (x) = A (u) (x - u), A (u) = (B (u)^{t})^{- 1} . \end{matrix}

(2.17)

Definition 2.11. 1) The coordinates provided by

ψ_{u}

are called the privileged coordinates at

u

with respect to the

H

-frame

X_{0}, \dots, X_{d}

2) The map

ψ_{u}

is called the privileged-coordinate map with respect to the

H

-frame

X_{0}, \dots, X_{d}

Remark 2.12. The privileged coordinates at

u

are called

u

-coordinates in [BG] , but they correspond to the privileged coordinates of [Be] and [Gro] in the special case of a Heisenberg manifold.

Next, on

R d + 1

we consider the dilations

\begin{matrix} δ_{t} (x) = t . x = (t^{2} x_{0}, t x_{1}, \dots, t x_{d}), t \in R, \end{matrix}

(2.18)

with respect to which

\frac{\partial}{\partial_{x_{0}}}

is homogeneous of degree

- 2

and

\frac{\partial}{\partial_{x_{1}}}, \dots, \frac{\partial}{\partial_{x_{d}}}

is homogeneous of degree

- 1

Since in the privileged coordinates at

u

we have

X_{j} (0) = \frac{\partial}{\partial x_{j}}

we can write

\begin{matrix} X_{j} = \frac{\partial}{\partial_{x_{j}}} + \sum_{k = 0}^{d} a_{j k} (x) \frac{\partial}{\partial_{x_{k}}}, j = 0, 1, \dots d, \end{matrix}

(2.19)

where the

a_{j k}

's are smooth functions such that

a_{j k} (0) = 0

. Therefore, we may define

\begin{matrix} \begin{matrix} (2.20) & X_{0}^{(u)} = {lim}_{t \to 0} t^{2} δ_{t}^{*} X_{0} = \frac{\partial}{\partial_{x_{0}}}, X_{j}^{(u)} = {lim}_{t \to 0} t^{- 1} δ_{t}^{*} X_{j} = \frac{\partial}{\partial_{x_{j}}} + \sum_{k = 1}^{d} b_{j k} x_{k} \frac{\partial}{\partial_{x_{0}}}, j = 1, \dots, d, \end{matrix} \end{matrix}

(2.21)

where for

j, k = 1, \dots, d

we have set

b_{j k} = \partial_{x_{k}} a_{j 0} (0)

Observe that

X_{0}^{(u)}

is homogeneous of degree

- 2

and

X_{1}^{(u)}, \dots, X_{d}^{(u)}

are homogeneous of degree

- 1

. Moreover, for

j, k = 1, \dots, d

we have

\begin{matrix} [X_{j}^{(u)}, X_{0}^{(u)}] = 0 and [X_{j}^{(u)}, X_{0}^{(u)}] = (b_{k j} - b_{j k}) X_{0}^{(u)} . \end{matrix}

(2.22)

Thus, the linear space spanned by

X_{0}^{(u)}, X_{1}^{(u)}, \dots, X_{d}^{(u)}

is a graded 2-step nilpotent Lie algebra

g^{(u)}

. In particular,

g^{(u)}

is the Lie algebra of left-invariant vector fields over the graded Lie group

G^{(u)}

consisting of

R d + 1

equipped with the grading ( 2.18 ) and the group law,

\begin{matrix} x . y = (x_{0} + \sum_{j, k = 1}^{d} b_{k j} x_{j} x_{k}, x_{1}, \dots, x_{d}) . \end{matrix}

(2.23)

Now, if near

a

we let

ℒ (X_{j}, X_{k}) = [X_{j}, X_{k}] = L_{j k} (x) X_{0} m o d H

, then we get

\begin{matrix} [X_{j}^{(u)}, X_{k}^{(u)}] = {lim}_{t \to 0} [t δ_{t}^{*} X_{j}, t δ_{t}^{*} X_{k}] = {lim}_{t \to 0} t^{2} δ_{t}^{*} (L_{j k} (\circ κ^{- 1} (x)) X_{0}) = L_{j k} (a) X_{0}^{(u)} . \end{matrix}

(2.24)

Comparing this with ( 2.16 ) and ( 2.22 ) then shows that

g^{(u)}

has the same the constant structures as those of

g_{a} M

, hence is isomorphic to it. Consequently, the Lie groups

G^{(u)}

and

G_{a} M

are isomorphic. In fact, as shown in [BG] and [Po2] , an explicit isomorphism is given by

\begin{matrix} φ_{u} (x_{0}, \dots, x_{d}) = (x_{0} - \frac{1}{4} \sum_{j, k = 1}^{d} (b_{j k} + b_{k j}) x_{j} x_{k}, x_{1}, \dots, x_{d}) . \end{matrix}

(2.25)

Definition 2.13. Let

ɛ_{u} = φ_{u} \circ ψ_{u}

. Then: 1) The new coordinates provided by

ɛ_{u}

are called Heisenberg coordinates at

u

with respect to the

H

-frame

X_{0}, \dots, X_{d}

2) The map

ɛ_{u}

is called the

u

-Heisenberg coordinate map.

Remark 2.14. The Heisenberg coordinates at

u

have been also considered in [BG] as a technical tool for inverting the principal symbol of a hypoelliptic sublaplacian.

Next, by [Po2,Lem. 1.17] we have

\begin{matrix} φ_{*} X_{0}^{(u)} = \frac{\partial}{\partial x_{0}} = X_{0}^{a} and φ_{*} X_{j}^{(u)} = \frac{\partial}{\partial x_{j}} - \frac{1}{2} \sum_{k = 1}^{d} L_{j k} x_{k} \frac{\partial}{\partial x_{0}} = X_{j}^{a}, j = 1, \dots, d . \end{matrix}

(2.26)

Since

φ_{u}

commutes with the Heisenberg dilations ( 2.18 ), using ( 2.20 )–( 2.21 ) we get

\begin{matrix} {lim}_{t \to 0} t^{2} δ_{t}^{*} φ_{u *} X_{0}^{(u)} = X_{0}^{a} and {lim}_{t \to 0} t δ_{t}^{*} φ_{u *} X_{j}^{(u)} = X_{j}^{a}, j = 1, \dots, d . \end{matrix}

(2.27)

In fact, as shown in [Po2] , in Heisenberg coordinates at

a

for any vector fields

X

t \to 0

we have

\begin{matrix} δ_{t}^{*} X = {\begin{matrix} t^{- 2} X^{a} + O (t^{- 1}) & if X (a) \in H_{a}, \\ t^{- 1} X^{a} + O (1) & otherwise . \end{matrix} \end{matrix}

(2.28)

Therefore, we obtain:

Proposition 2.15 ([Po2] ). In the Heisenberg coordinates centered at

m = κ^{- 1} (u)

the tangent Lie group

G_{a} M

coincides with

G^{(u)}

and for any vector fields

X

the model vector fields

X^{a}

approximates

X

near

a

in the sense of ( 2.28 ).

One consequence of the equivalence between the two approaches to

G M

is a tangent approximation for Heisenberg diffeomorphisms as follows.

Let

φ : (M, H) \to (M^{'}, H^{'})

be a Heisenberg diffeomorphism from

(M, H)

to another Heisenberg manifold

(M^{'}, H^{'})

. We also endow

R d + 1

with the pseudo-norm,

\begin{matrix} ∥ x ∥ = (x_{0}^{2} + (x_{1}^{2} + \dots + x_{d}^{2})^{2})^{1 / 4}, x \in R d + 1, \end{matrix}

(2.29)

so that, for any

x \in R d + 1

and any

t \in R

, we have

\begin{matrix} ∥ t . x ∥ = | t | ∥ x ∥ . \end{matrix}

(2.30)

Proposition 2.16 ([Po2,Prop. 2.21] ). Let

a \in M

and set

m^{'} = φ (a)

. Then, in Heisenberg coordinates at

a

and at

a^{'}

the diffeomorphism

φ (x)

has a behavior near

x = 0

of the form

\begin{matrix} φ (x) = φ_{H}^{'} (0) x + (O (∥ x ∥^{3}), O (∥ x ∥^{2}), \dots, O (∥ x ∥^{2})) . \end{matrix}

(2.31)

In particular, there are no terms of the form

x_{j} x_{k}

1 \leq j, k \leq d

, in the Taylor expansion of

φ_{0} (x)

x = 0

Remark 2.17. An asymptotics similar to ( 2.31 ) is given in [Be,Prop. 5.20] in privileged coordinates at

u

and

u^{'} = κ_{1} (a^{'})

, but the leading term there is only a Lie algebra isomorphism from

g^{(u)}

onto

g^{(u^{'})}

. This is only in Heisenberg coordinates that we recover the Lie group isomorphism

φ_{H}^{'} (a)

as the leading term of the asymptotics.

3 Hypoelliptic calculus on Heisenberg manifolds

The Heisenberg calculus is the relevant pseudodifferential tool to study hypoelliptic operators on Heisenberg manifolds. It was independently invented by Beals-Greiner [BG] and Taylor [Tay] , extending previous works of Boutet de Monvel [Bo1] , Folland-Stein [FS] and Dynin ([Dy1] , [Dy2] ) (see also [BGH] , [CGGP] , [EMM] , [Gri] , [H¨3] , [RSt] ). The idea in the Heisenberg calculus is to have a pseudodifferential calculus on a Heisenbergmanifold

(M, H)

which is modeled at any point

a \in M

by the calculus of left-invariant pseudodifferentialoperators on the tangent group

G_{a} M

3.1 Left-invariant pseudodifferential operators

Let

(M^{d + 1}, H)

be a Heisenberg manifold and let

G = G_{a} M

be the tangent Lie group of

M

at a point

a \in M

. We recall here the main facts about left-invariant pseudodifferential operators on

G

(see also [BG] , [CGGP] , [Tay] ).

Recall that for any finite dimensional vector space

E

the Schwartz class

S (E)

is a Fréchet space and the Fourier transform is the continuous isomorphism of

S (E)

onto

S (E^{*})

given by

\begin{matrix} \hat{f} (ξ) = \int_{E} e^{i 〈 ξ, x 〉} f (x) d x, f \in S (E), ξ \in E^{*}, \end{matrix}

(3.1)

where

d x

denotes the Lebesgue measure of

E

Definition 3.1.

S_{0} (E)

is the closed subspace of

S (E)

consisting of

f \in S (E)

such that for any differential operator

P

E^{*}

we have

(P \hat{f}) (0) = 0

Since

G

has the same underlying set as that of its Lie algebra

g = g_{x} M

we can let

S (G)

and

S_{0} (G)

denote the Fréchet spaces

S (E)

and

S_{0} (E)

associated to the underlying linear space

E

g

(notice that the Lebesgue measure of

E

coincides with the Haar measure of

G

since

G

is nilpotent).

Next, for

λ \in R

and

ξ = ξ_{0} + ξ^{'}

g^{*} = (T_{a}^{*} M / H_{a}^{*}) \oplus H_{a}

we let

\begin{matrix} λ . ξ = λ . (ξ_{0} + ξ^{'}) = λ^{2} ξ_{0} + λ ξ^{'} . \end{matrix}

(3.2)

Definition 3.2.

S_{m} (g^{*})

m \in C

, is the space of functions

p \in C^{\infty} (g^{*} \ 0)

which are homogeneous of degree

m

, in the sense that, for any

λ > 0

we have

\begin{matrix} p (λ . ξ) = λ^{m} p (ξ) \forall ξ \in g^{*} \ 0 . \end{matrix}

(3.3)

In addition

S_{m} (g^{*})

is endowed with the Fréchet space topology induced from that of

C^{\infty} (g^{*} \ 0)

Note that the image

{\hat{S}}_{0} (G)

S (G)

under the Fourier transform consists of functions

v \in S (g^{*})

such that, given any norm

| . |

G

, near

ξ = 0

we have

| g (ξ) | = O (| ξ |^{N})

for any integer

N \geq 0

Thus, any

p \in S_{m} (g^{*})

defines an element of

{\hat{S}}_{0} (g^{*})^{'}

by letting

\begin{matrix} 〈 p, g 〉 = \int_{g^{*}} p (ξ) g (ξ) d ξ, g \in {\hat{S}}_{0} (g^{*}) . \end{matrix}

(3.4)

This allows us to define the inverse Fourier transform of

p

as the element

\overset{ˇ}{p} \in S_{0} (G)^{'}

such that

\begin{matrix} 〈 \overset{ˇ}{p}, f 〉 = 〈 p, \overset{ˇ}{f} 〉 \forall f \in S_{0} (G) . \end{matrix}

(3.5)

Proposition 3.3 ([BG] , [CGGP] ). 1) For any

p \in S_{m} (g^{*})

the left-convolution by

\overset{ˇ}{p}

\begin{matrix} \overset{ˇ}{p} * f (x) : = 〈 \overset{ˇ}{p} (y), f (x . y^{- 1}) 〉, f \in S_{0} (G), \end{matrix}

(3.6)

defines a continuous endomorphism of

S_{0} (G)

2) There is a continuous bilinear product,

\begin{matrix} * : S_{m_{1}} (g^{*}) \times S_{m_{2}} (g^{*}) ⟶ S_{m_{1} + m_{2}} (g^{*}), \end{matrix}

(3.7)

such that, for any

p_{1} \in S_{m_{1}} (g^{*})

and

p_{2} \in S_{m_{2}} (g^{*})

, the composition of the left-convolution operators by

{\overset{ˇ}{p}}_{1}

and

{\overset{ˇ}{p}}_{2}

is the left-convolution operator by

(p_{1} * p_{2})^{\lor}

, that is, we have

\begin{matrix} {\overset{ˇ}{p}}_{1} * ({\overset{ˇ}{p}}_{2} * f) = (p_{1} * p_{2})^{\lor} * f \forall f \in S_{0} (G) . \end{matrix}

(3.8)

Let us also mention that if

p \in S_{m} (g^{*})

then the convolution operator

P u = \overset{ˇ}{p} * f

is a pseudodifferential operator. Indeed, let

X_{0} (a), \dots, X_{d} (a)

be a (linear) basis of

g

so that

X_{0} (a)

is in

T_{a} M / H_{a}

and

X_{1} (a), \dots, X_{d} (a)

span

H_{a}

. For

j = 0, \dots, d

let

X_{j}^{a}

be the left-invariant vector fields on

G

such that

X_{j |_{x = 0}}^{w} = X_{j} (a)

. The basis

X_{0} (a), \dots, X_{d} (a)

yields a linear isomorphism

g ≃ R d + 1

, hence a global chart of

G

. In this chart

p

is a homogeneous symbol on

R d + 1 \ 0

with respect to the dilations

\begin{matrix} λ . x = (λ^{2} x_{0}, λ x_{1}, \dots, λ x_{d}), x \in R d + 1, λ > 0 . \end{matrix}

(3.9)

Similarly, each vector fields

\frac{1}{i} X_{j}^{a}

j = 0, \dots, d

, corresponds to a vector fields on

R d + 1

whose symbol is denoted

σ_{j}^{a} (x, ξ)

. Then, setting

σ = (σ_{0}, \dots, σ_{d})

, it can be shown that in the above chart the operator

P

is given by

\begin{matrix} P f (x) = \int_{R d + 1} e^{i x . ξ} p (σ^{a} (x, ξ)) \hat{f} (ξ), f \in S_{0} (R d + 1) . \end{matrix}

(3.10)

In other words

P

is the pseudodifferential operator

p (- i X^{a}) : = p (σ^{a} (x, D))

acting on

S_{0} (R d + 1)

3.2 $Ψ_{H}$ DO's on an open subset of $R d + 1$

Let

U

be an open subset of

R d + 1

together with a hyperplane bundle

H \subset T U

and a

H

-frame

X_{0}, X_{1}, \dots, X_{d}

T U

. Then the class of

Ψ H

DO's on

U

is a class of pseudodifferential operators modelled on that of homogeneous convolution operators on the fibers of

G U

Definition 3.4.

S_{m} (U \times R d + 1)

m \in C

, is the space of symbols

p (x, ξ) \in C^{\infty} (U \times R d + 1 \ 0)

that are homogeneous of degree

m

with respect to the

ξ

-variable, that is,

\begin{matrix} p (x, λ . ξ) = λ^{m} p (x, ξ) for any λ > 0, \end{matrix}

(3.11)

where

ξ \to λ . ξ

denotes the Heisenberg dilation ( 3.9 ).

Observe that the homogeneity of

p \in S_{m} (U \times R d + 1)

implies that, for any compact

K \subset U

, it satisfies the estimates

\begin{matrix} | \partial_{x}^{α} \partial_{ξ}^{β} p (x, ξ) | \leq C_{K α β} ∥ ξ ∥^{ℜ m - 〈 β 〉}, x \in K, ξ \neq 0, \end{matrix}

(3.12)

where

∥ ξ ∥ = (| ξ_{0} |^{2} + | ξ_{1} |^{4} + \dots + | ξ_{d} |^{4})^{1 / 4}

and

〈 α 〉 = 2 α_{0} + α_{1} + \dots + α_{d}

Definition 3.5.

S^{m} (U \times R d + 1)

m \in C

, consists of symbols

p \in C^{\infty} (U \times R d + 1)

with an asymptotic expansion

p \sim \sum_{j \geq 0} p_{m - j}

p_{k} \in S_{k} (U \times R d + 1)

, in the sense that, for any integer

N

and for any compact

K \subset U

, we have

\begin{matrix} | \partial_{x}^{α} \partial_{ξ}^{β} (p - \sum_{j < N} p_{m - j}) (x, ξ) | \leq C_{α β N K} ∥ ξ ∥^{ℜ m - 〈 β 〉 - N}, x \in K, ∥ ξ ∥ \geq 1 . \end{matrix}

(3.13)

Next, for

j = 0, \dots, d

let

σ_{j} (x, ξ)

denote the symbol of

\frac{1}{i} X_{j}

(in the classical sense) and set

σ = (σ_{0}, \dots, σ_{d})

. For any

p \in S^{m} (U \times R d + 1)

it can be shown that the symbol

p_{σ} (x, ξ) : = p (x, σ (x, ξ))

is in the Hörmander class of symbols of type

(\frac{1}{2}, \frac{1}{2})

(see [BG,Prop. 10.22] ). Therefore, we define a continuous linear operator from

C_{c}^{\infty} (U)

C^{\infty} (U)

by letting

\begin{matrix} p (x, - i X) f (x) = (2 π)^{- (d + 1)} \int e^{i x . ξ} p (x, σ (x, ξ)) \hat{f} (ξ) d ξ, f \in C_{c}^{\infty} (U) . \end{matrix}

(3.14)

In the sequel we let

Ψ^{- \infty} (U)

denotes the class of smoothing operators, i.e. of operators given by smooth kernels.

Definition 3.6.

Ψ^{m} H (U)

m \in C

, consists of operators

P : C_{c}^{\infty} (U) \to C^{\infty} (U)

of the form

\begin{matrix} P = p (x, - i X) + R, \end{matrix}

(3.15)

with

p

S^{m} (U \times R d + 1)

, called the symbol of

P

, and with

R

Ψ^{- \infty} (U)

The above definition of the symbol of

P

differs from that of [BG] , since there the authors defined it to be

p_{σ} (x, ξ) = p (x, σ (x, ξ))

. Note also that

p

is unique modulo

S^{- \infty} (U \times R d + 1)

Lemma 3.7. For

j = 0, 1, \dots

let

p_{m - j} \in S_{m - j} (U \times R d + 1)

. Then there exists

P \in Ψ^{m} H (U)

with symbol

p \sim \sum_{j \geq 0} p_{m - j}

. Moreover, the operator

P

is unique modulo smoothing operators.

The class

Ψ^{m} H (U)

does not depend on the choice of the

H

-frame

X_{0}, \dots, X_{d}

(see [BG,Prop. 10.46] ).

Moreover, since it is contained in the class of

Ψ

DO's of type

(\frac{1}{2}, \frac{1}{2})

we get:

Proposition 3.8. Let

P

be a

Ψ H

DO of order

m

U

P

extends to a continuous linear mapping from

ℰ^{'} (U)

D^{'} (U)

and has a distribution kernel which is smooth off the diagonal.

3) Let

k = ℜ m

ℜ m \geq 0

and

k = \frac{1}{2} ℜ m

otherwise. Then for any

s \in R

the operator

P \in Ψ^{m} H (U)

extends to a continuos mapping from

L_{s, comp}^{2} (U)

L_{s - k, loc}^{2} (U)

3.3 Composition of $Ψ_{H}$ DO's

Recall that there is no symbolic calculus for

Ψ

DO's of type

(\frac{1}{2}, \frac{1}{2})

since the product of two such

Ψ

DO's needs not be again a

Ψ

DO of type

(\frac{1}{2}, \frac{1}{2})

. However, the fact that the

Ψ H

DO's are modelled on left-invariant pseudodifferential operators allows us to construct a symbolic calculus for

Ψ H

DO's. First, for

j = 0, \dots, d

let

X_{j}^{(x)}

be the leading homogeneous part of

X_{j}

in privileged coordinates centered at

x

defined according to ( 2.20 )–( 2.21 ). These vectors span a nilpotent Lie algebra of left-invariant vector fields on a nilpotent graded Lie group

G^{x}

which corresponds to

G_{x} U

by pulling back the latter from the Heisenberg coordinates at

x

to the privileged coordinates at

x

As alluded to above the product law of

G^{(x)}

defines a convolution product for symbols,

\begin{matrix} *^{(x)} : S_{m_{1}} (R d + 1) \times S_{m_{2}} (R d + 1) ⟶ S_{m_{1} + m_{2}} (R d + 1) . \end{matrix}

(3.16)

such that, with the notations of ( 3.10 ), on

ℒ (S_{0} (R d + 1))

we have

\begin{matrix} p_{1} (- i X^{(x)}) p_{2} (- i X^{(x)}) = (p_{1} *^{(x)} p_{2}) (- i X^{(x)}) \forall p_{j} \in S_{m_{j}} (R d + 1) . \end{matrix}

(3.17)

As it turns out the product

*^{(x)}

depends smoothly on

x

(see [BG,Prop. 13.33] ). Therefore, we get a continuous bilinear product,

\begin{matrix} \begin{matrix} (3.18) & * : S_{m_{1}} (U \times R d + 1) \times S_{m_{2}} (U \times R d + 1) \to S_{m_{1} + m_{2}} (U \times R d + 1), p_{1} * p_{2} (x, ξ) = (p_{1} (x, .) *^{(x)} p_{2} (x, .)) (ξ), p_{j} \in S_{m_{j}} (U \times R d + 1) . \end{matrix} \end{matrix}

(3.19)

Proposition 3.9 ([BG,Thm. 14.7] ). For

j = 1, 2

let

P_{j} \in Ψ^{m_{j}} H (U)

have symbol

p_{j} \sim \sum_{k \geq 0} p_{j, m_{j} - k}

and assume that one of these operators is properly supported. Then the operator

P = P_{1} P_{2}

is a

Ψ H

DO of order

m_{1} + m_{2}

and has symbol

p \sim \sum_{k \geq 0} p_{m_{1} + m_{2} - k}

, with

\begin{matrix} p_{m_{1} + m_{2} - k} (x, ξ) = \sum_{k_{1} + k_{2} \leq k} \sum_{α, β, γ, δ}^{(k - k_{1} - k_{2})} h_{α β γ δ} (x) (D_{ξ}^{δ} p_{1, m_{1} - k_{1}}) * (ξ^{γ} \partial_{x}^{α} \partial_{ξ}^{β} p_{2, m_{2} - k_{2}}) (x, ξ), \end{matrix}

(3.20)

where

\sum (l) α β γ δ

denotes the sum over the indices such that

| β | = | γ |

and

| α | + | β | \leq 〈 β 〉 - 〈 γ 〉 + 〈 δ 〉 = l

, and the functions

h_{α β γ δ} (x)

's are polynomials in the derivatives of the coefficients of the vector fields

X_{0}, \dots, X_{d}

3.4 The distribution kernels of $Ψ_{H}$ DO's

An important fact about

Ψ

DO's is their characterization in terms of their distribution kernels.

First, we extend the notion of homogeneity of functions to distributions. For

K \in S^{'} (R d + 1)

and for

λ > 0

we let

K_{λ}

denote the element of

S^{'} (R d + 1)

such that

\begin{matrix} 〈 K_{λ}, f 〉 = λ^{- (d + 2)} 〈 K (x), f (λ^{- 1} . x) 〉 \forall f \in S (R d + 1) . \end{matrix}

(3.21)

In the sequel we will also use the notation

K (λ . x)

for denoting

K_{λ} (x)

. We then say that

K

is homogeneous of degree

m

m \in C

, when

K_{λ} = λ^{m} K

for any

λ > 0

Definition 3.10.

S_{reg}^{'} (R d + 1)

consists of tempered distributions on

R d + 1

which are smooth outside the origin. We equip it with the weakest topology such that the inclusions of

S_{reg}^{'} (R d + 1)

into

S^{'} (R d + 1)

and

C^{\infty} (R d + 1 \ 0)

are continuous.

Definition 3.11.

K_{m} (U \times R d + 1)

m \in C

, consists of distributions

K (x, y)

C^{\infty} (U) \hat{\otimes} S_{reg}^{'} (R d + 1)

such that for some functions

c_{α} (x) \in C^{\infty} (U)

〈 α 〉 = m

, we have

\begin{matrix} K (x, λ . y) = λ^{m} K (x, y) + λ^{m} log λ \sum_{〈 α 〉 = m} c_{α} (x) y^{α} for any λ > 0 . \end{matrix}

(3.22)

The interest of considering the distribution class

K_{m} (U \times R d + 1)

stems from:

Lemma 3.12 ([BG,Prop. 15.24] , [CM,Lem. I.4] ). 1) Any

p \in S_{m} (U \times R d + 1)

agrees on

U \times (R d + 1 \ 0)

with a distribution

τ (x, ξ) \in C^{\infty} (U) \hat{\otimes} S^{'} (R d + 1)

such that

{\overset{ˇ}{τ}}_{ξ \to y}

is in

K_{\hat{m}} (U \times R d + 1)

\hat{m} = - (m + d + 2)

2) If

K (x, y)

is in

K_{\hat{m}} (U \times R d + 1)

then the restriction of

{\hat{K}}_{y \to ξ} (x, ξ)

U \times (R d + 1 \ 0)

belongs to

S_{m} (U \times R d + 1)

This result is a consequence of the solution to the problem of extending a homogeneous function

p \in C^{\infty} (R d + 1 \ 0)

into a homogeneous distribution on

R d + 1

and of the fact that for

τ \in S^{'} (R d + 1)

we have

\begin{matrix} (\hat{τ})_{λ} = | λ |^{- (d + 2)} (τ_{λ^{- 1}})^{\land} \forall λ \in R \ 0 . \end{matrix}

(3.23)

In particular, if

τ

is homogeneous of degree

m

then

\hat{τ}

is homogeneous of degree

- (m + d + 2)

The relevant class of kernels for the Heisenberg calculus is the following.

Definition 3.13.

K^{m} (U \times R d + 1)

m \in C

, consists of distributions

K \in D^{'} (U \times R d + 1)

with an asymptotic expansion

K \sim \sum_{j \geq 0} K_{m + j}

K_{l} \in K_{l} (U \times R d + 1)

, in the sense that, for any integer

N

, as soon as

J

is large enough we have

\begin{matrix} K - \sum_{j \leq J} K_{m + j} \in C^{N} (U \times R d + 1) . \end{matrix}

(3.24)

Since under the Fourier transform the asymptotic expansion ( 3.13 ) for symbols corresponds to that for distributions in ( 3.24 ), using Lemma 3.12 we get:

Lemma 3.14 ([BG,pp. 133–134] ). Let

K \in D^{'} (U \times R d + 1)

. Then the following are equivalent:

(i) The distribution

K

belongs to

K^{m} (U \times R d + 1)

; (ii) We can put

K

into the form

\begin{matrix} K (x, y) = {\overset{ˇ}{p}}_{ξ \to y} (x, y) + R (x, y), \end{matrix}

(3.25)

for some

p \in S^{\hat{m}} (U \times R d + 1)

\hat{m} = - (m + d + 2)

, and some

R \in C^{\infty} (U \times R d + 1)

Moreover, if (i) and (ii) holds and we expand

K \sim \sum_{j \geq 0} K_{m + j}

K_{l} \in K_{l} (U \times R d + 1)

, then we have

p \sim \sum_{j \geq 0} p_{\hat{m} - j}

where

p_{\hat{m} - j} \in S_{\hat{m} - j} (U \times R d + 1)

is the restriction to

U \times (R d + 1 \ 0)

(K_{m + j})_{y \to ξ}^{\land}

Next, for

x \in U

let

ψ_{x}

denote the affine change to the privileged coordinates at

x

and let us write

(A_{x}^{t})^{- 1} ξ = σ (x, ξ)

with

A_{x} \in {GL}_{d + 1} (R)

. Since

ψ_{x} (x) = 0

and

ψ_{x *} X_{j} = \frac{\partial}{\partial y_{j}}

y = 0

for

j = 0, \dots, d

, one checks that

ψ_{x} (y) = A_{x} (y - x)

Let

p \in S^{m} (U \times R d + 1)

. As

p (x, - i X) = p_{σ} (x, D)

with

p_{σ} (x, ξ) = p (x, σ (x, ξ)) = p (x, (A_{x}^{t})^{- 1} ξ)

the distribution kernel

k_{p (x, - i X)} (x, y)

p (x, - i X)

is represented by the oscillating integrals

\begin{matrix} (2 π)^{- (d + 1)} \int e^{i (x - y) . ξ} p (x, (A_{x}^{t})^{- 1} ξ) d ξ = (2 π)^{- (d + 1)} | A_{x} | \int e^{i A_{x} (x - y) . ξ} p (x, ξ) d ξ . \end{matrix}

(3.26)

Since

ψ_{x} (y) = A_{x} (y - x)

we deduce that

\begin{matrix} k_{p (x, - i X)} (x, y) = | ψ_{x}^{'} | {\overset{ˇ}{p}}_{ξ \to y} (x, - ψ_{x} (y)) . \end{matrix}

(3.27)

Combining this with Lemma 3.14 then gives:

Proposition 3.15 ([BG,Thms. 15.39,15.49] ). Let

P : C_{c}^{\infty} (U) \to C^{\infty} (U)

be a continuous linear operator with distribution kernel

k_{P} (x, y)

. Then the following are equivalent:

(i)

P

is a

Ψ H

DO of order

m

m \in C

(ii) There exist

K \in K^{\hat{m}} (U \times R d + 1)

\hat{m} = - (m + d + 2)

, and

R \in C^{\infty} (U \times U)

such that

\begin{matrix} k_{P} (x, y) = | ψ_{x}^{'} | K (x, - ψ_{x} (y)) + R (x, y) . \end{matrix}

(3.28)

Furthermore, if (i) and (ii) hold and

K \sim \sum_{j \geq 0} K_{\hat{m} + j}

K_{l} \in K_{l} (U \times R d + 1)

, then

P

has symbol

p \sim \sum_{j \geq 0} p_{m - j}

p_{l} \in S_{l} (U \times R d + 1)

, where

p_{m - j}

is the restriction to

U \times (R d + 1 \ 0)

(K_{m + j})_{y \to ξ}^{\land}

In the sequel we will need a version of Proposition 3.15 in Heisenberg coordinates. To this end let

ɛ_{x}

denote the coordinate change to the Heisenberg coordinates at

x

and set

φ_{x} = ɛ_{x} \circ ψ_{x}^{- 1}

Recall that

φ_{x}

is a Lie group isomorphism from

G^{(x)}

G_{x} U

such that

φ_{x} (λ . y) = λ . φ_{x} (y)

for any

λ \in R

. Moreover, using ( 2.25 ) one can check that

| φ_{x}^{'} | = 1

and

φ_{x}^{- 1} (y) = - φ_{x} (- y)

. Therefore, from ( 3.27 ) we see that we can put

k_{p (x, - i X)} (x, y)

into the form

\begin{matrix} k_{p (x, - i X)} (x, y) = | ɛ_{x}^{'} | K_{P} (x, - ɛ_{x} (y)), K_{P} (x, y) = {\overset{ˇ}{p}}_{ξ \to y} (x, - φ_{x} (- y)) = {\overset{ˇ}{p}}_{ξ \to y} (x, φ_{x}^{- 1} (y)) . \end{matrix}

(3.29)

In fact, the coordinate changes

φ_{x}

give rise to an action on distributions on

U \times R d + 1

given by

\begin{matrix} K (x, y) ⟶ φ_{x}^{*} K (x, y), φ_{x}^{*} K (x, y) = K (x, φ_{x}^{- 1} (y)) . \end{matrix}

(3.30)

Since

φ_{x}

depends smoothly on

x

, this action induces a continuous linear isomorphisms of

C^{N} (U \times R d + 1)

N \geq 0

, and

C^{\infty} (U \times R d + 1)

onto themselves. As

φ_{x} (y)

is polynomial in

y

in such way that

φ_{x} (0) = 0

and

φ_{x} (λ . y) = λ . φ_{x} (y)

for every

λ \in R

, we deduce that the above action also yields a continuous linear isomorphism of

C^{\infty} (U) \hat{\otimes} S_{reg}^{'} (R d + 1)

onto itself and, for every

λ > 0

, we have

\begin{matrix} (φ_{x}^{*} K) (x, λ . y) = φ_{x}^{*} [K (x, λ . y)], K \in D^{'} (U \times R d + 1) . \end{matrix}

(3.31)

Furthermore, as

φ_{x} (y)

is polynomial in

y

we see that for every

α \in N^{d + 1}

we can write

φ_{x} (y)^{α}

in the form

φ_{x} (y)^{α} = \sum_{〈 β 〉 = 〈 α 〉} d_{α β} (x) y^{β}

with

d_{α β} \in C^{\infty} (U \times R d + 1)

. It then follows that, for every

m \in C

, the map

K (x, y) \to φ_{x}^{*} K (x, y)

induces a linear isomorphisms of

K_{m} (U \times R d + 1)

and

K^{m} (U \times R d + 1)

onto themselves. Combining this with ( 3.29 ) and Proposition 3.15 then gives:

Proposition 3.16. Let

P : C_{c}^{\infty} (U) \to C^{\infty} (U)

be a continuous linear operator with distribution kernel

k_{P} (x, y)

. Then the following are equivalent:

(i)

P

is a

Ψ H

DO of order

m

m \in C

(ii) There exist

K_{P} \in K^{\hat{m}} (U \times R d + 1)

\hat{m} = - (m + d + 2)

, and

R \in C^{\infty} (U \times U)

such that

\begin{matrix} k_{P} (x, y) = | ɛ_{x}^{'} | K_{P} (x, - ɛ_{x} (y)) + R (x, y) . \end{matrix}

(3.32)

Furthermore, if (i) and (ii) hold and

K_{P} \sim \sum_{j \geq 0} K_{P, \hat{m} + j}

K_{l} \in K_{l} (U \times R d + 1)

, then

P

has symbol

p \sim \sum_{j \geq 0} p_{m - j}

p_{l} \in S_{l} (U \times R d + 1)

, where

p_{m - j}

is the restriction to

U \times (R d + 1 \ 0)

[K_{P, \hat{m} + j} (x, φ_{x}^{- 1} (y))]_{y \to ξ}^{\land}

Remark 3.17. Let

a \in U

. Then ( 3.32 ) shows that the distribution kernel of

\tilde{P} = (ɛ_{a})_{*} P

x = 0

\begin{matrix} k_{\tilde{P}} (0, y) = | ɛ_{a}^{'} |^{- 1} k_{P} (ɛ_{a}^{- 1} (0), ɛ_{a}^{- 1} (y)) = K_{P} (a, - y) . \end{matrix}

(3.33)

Moreover, as we are in Heisenberg coordinates already, we have

ψ_{0} = ɛ_{0} = φ_{0} = id

. Thus, in the form ( 3.32 ) for

\tilde{P}

we have

K_{\tilde{P}} (0, y) = K_{P} (a, y)

. Therefore, if we let

p_{m} (x, ξ)

denote the principal symbol of

P

and let

K_{P, \hat{m}} \in K_{\hat{m}} (U \times R d + 1)

denote the leading kernel of

K_{P}

, then by Proposition 3.16 we have

\begin{matrix} p_{m} (0, ξ) = [K_{P, \hat{m}}]_{y \to ξ}^{\land} (a, ξ) . \end{matrix}

(3.34)

This shows that

[K_{P, \hat{m}}]_{y \to ξ}^{\land} (a, ξ)

is the principal symbol of

P

x = 0

in Heisenberg coordinates centered at

a

3.5 $Ψ_{H}$ DO's on a general Heisenberg manifold

Let

(M^{d + 1}, H)

be a Heisenberg manifold. As alluded to before the

Ψ H

DO's on an subset of

R d + 1

are

Ψ

DO's of type

(\frac{1}{2}, \frac{1}{2})

. However, the latter don't make sense on a general manifold, for their class is not preserved by an arbitrary change of chart. Nevertheless, when dealing with

Ψ H

DO's this issue is resolved if we restrict ourselves to changes of Heisenberg charts. Indeed, we have:

Proposition 3.18. Let

U

(resp.

\tilde{U}

) be an open subset of

R d + 1

together with a hyperplane bundle

H \subset T U

(resp.

\tilde{H} \subset T \tilde{U}

) and a

H

-frame of

T U

(resp. a

\tilde{H}

-frame of

T \tilde{U}

). Let

φ : (U, H) \to (\tilde{U}, \tilde{H})

be a Heisenberg diffeormorphism and let

\tilde{P} \in Ψ_{\tilde{H}}^{m} (\tilde{U})

1) The operator

P = φ^{*} \tilde{P}

is a

Ψ H

DO of order

m

U

2) If the distribution kernel of

\tilde{P}

is of the form ( 3.32 ) with

K_{\tilde{P}} (\tilde{x}, \tilde{y}) \in K^{\hat{m}} (\tilde{U} \times R d + 1)

then the distribution kernel of

P

can be written in the form ( 3.32 ) with

K_{P} (x, y) \in K^{\hat{m}} (U \times R d + 1)

such that

\begin{matrix} K_{P} (x, y) \sim \sum_{〈 β 〉 \geq \frac{3}{2} 〈 α 〉} \frac{1}{α! β!} a_{α β} (x) y^{β} (\partial_{\tilde{y}}^{α} K_{\tilde{P}}) (φ (x), φ_{H}^{'} (x) y), \end{matrix}

(3.35)

where we have let

a_{α β} (x) = \partial_{y}^{β} [| \partial_{y} ({\tilde{ɛ}}_{φ (x)} \circ φ \circ ɛ_{x}^{- 1}) (y) | ({\tilde{ɛ}}_{φ (x)} \circ φ \circ ɛ_{x}^{- 1} (y) - φ_{H}^{'} (x) y)^{α}]_{|_{y = 0}}

and

{\tilde{ɛ}}_{\tilde{x}}

denote the change to the Heisenberg coordinates at

\tilde{x} \in \tilde{U}

. In particular, we have

\begin{matrix} K_{P} (x, y) = | φ_{H}^{'} (x) | K_{\tilde{P}} (φ (x), φ_{H}^{'} (x) y) m o d K^{\hat{m} + 1} (U \times R d + 1) . \end{matrix}

(3.36)

Remark 3.19. The version of the above statement in [BG] does not contain the asymptotics ( 3.36 ), which will be crucial for giving a global definition of the principal symbol of a

Ψ H

DO in the next section. For this reason give a detailed proof of the above version in Section 8 . This proof will also be useful in [Po3] and [Po4] for generalizing Proposition 3.18 to holomorphic families of

Ψ H

DO's and to

Ψ H

DO's with parameter.

As a consequence of Proposition 3.18 we can define

Ψ H

DO's on

M

acting on the sections of a vector bundle

ℰ

over

M

Definition 3.20.

Ψ^{m} H (M, ℰ)

m \in C

, consists of continuous operators

P : C_{c}^{\infty} (M, ℰ) \to C^{\infty} (M, ℰ)

such that:

(i) The distribution kernel of

P

is smooth off the diagonal; (ii) For any trivialization

τ : ℰ_{|_{U}} \to U \times C^{r}

over a local Heisenberg chart

κ : U \to V \subset R d + 1

the operator

κ_{*} τ_{*} (P_{|_{U}})

belongs to

Ψ^{m} H (V, C^{r}) : = Ψ^{m} H (V) \otimes End C^{r}

All the previous properties of

Ψ

DO's on an open subset of

R d + 1

hold mutatis standis for

Ψ H

DO's on

M

acting on sections of

ℰ

3.6 Transposes and adjoints of $Ψ_{H}$ DO's

Let us now look at the transpose and adjoints of

Ψ H

DO's. First, given a Heisenberg chart

U \subset R d + 1

we have:

Proposition 3.21. Let

P \in Ψ^{m} H (U)

. Then: 1) The transpose operator

P^{t}

is a

Ψ H

DO of order

m

U

2) If we write the distribution kernel of

P

in the form ( 3.32 ) with

K_{P} \in K^{\hat{m}} (U \times R d + 1)

then

P^{t}

can be written in the form ( 3.32 ) with

K_{P^{t}} \in K^{\hat{m}} (U \times R d + 1)

such that

\begin{matrix} K_{P^{t}} (x, y) \sim \sum_{\frac{3}{2} 〈 α 〉 \leq 〈 β 〉} \sum_{| γ | \leq | δ | \leq 2 | γ |} a_{α β γ δ} (x) y^{β + δ} (\partial_{x}^{γ} \partial_{y}^{α} K_{P}) (x, - y), \end{matrix}

(3.37)

where

a_{α β γ δ} (x) = \frac{| ɛ_{x}^{- 1} |}{α! β! γ! δ!} [\partial_{y}^{β} (| ɛ_{ɛ_{x}^{- 1} (- y)}^{'} | (y - ɛ_{ɛ_{x}^{- 1} (y)} (x))^{α}) \partial_{y}^{δ} (ɛ_{x}^{- 1} (- y) - x)^{γ}] (x, 0)

. In particular,

\begin{matrix} K_{P^{t}} (x, y) = K_{P} (x, - y) m o d K^{\hat{m} + 1} (U \times R d + 1) . \end{matrix}

(3.38)

Remark 3.22. The asymptotic expansion ( 3.37 ) is not stated in [BG] , but we need it in order to determine the global principal symbol of the transpose of a

Ψ H

DO (see next section). A detailed proof of Proposition 3.21 can be found in Section 9 .

Using this result, or its version in [BG] , we obtain:

Proposition 3.23 ([BG,Thm. 17.4] ). Let

P : C^{\infty} (M, ℰ) \to C^{\infty} (M, ℰ)

be a

Ψ H

DO of order

m

. Then: 1) The transpose operator

P^{t} : ℰ^{'} (M, ℰ^{*}) \to D^{'} (M, ℰ^{*})

is a

Ψ H

DO of order

m

; 2) If

M

is endowed with a smooth positive density and

ℰ

with a Hermitian metric then the adjoint operator

P^{*} : C^{\infty} (M, ℰ) \to C^{\infty} (M, ℰ)

is a

Ψ H

DO of order

\bar{m}

4 Principal symbol and model operators.

In this section we define the principal symbols and model operators of

Ψ H

DO's and check their main properties.

4.1 Principal symbol and model operators

Let

g^{*} M

be the dual bundle of

g M

with canonical projection

π : g^{*} M \to M

Definition 4.1.

S_{m} (g^{*} M, ℰ)

m \in C

, is the space of sections

p \in C^{\infty} (g^{*} M \ 0, End π^{*} ℰ)

which are homogeneous of degree

m

in the sense that, for any

λ > 0

, we have

\begin{matrix} p (x, λ . ξ) = λ^{m} p (x, ξ) \forall (x, ξ) \in g^{*} M \ 0, \end{matrix}

(4.1)

where

ξ \to λ . ξ

denotes the dilation ( 3.2 ).

Let

P \in Ψ^{m} H (M)

and for

j = 1, 2

let

κ_{j}

be a Heisenberg chart with domain

V_{j} \subset M

and let

φ : U_{1} \to U_{2}

be the corresponding transition map, where we have let

U_{j} = κ_{j} (V_{1} \cap V_{2}) \subset R d + 1

Let us first assume that

ℰ

is the trivial line bundle, so that

P

is a scalar operator. For

j = 1, 2

we let

P_{j} : = κ_{j *} (P_{|_{V_{1} \cap V_{2}}})

, so that

P_{1} = φ^{*} P_{2}

. Since

P_{j}

belongs to

Ψ^{m} H (U_{j})

its distribution kernel is of the form ( 3.32 ) with

K_{P_{j}} \in K^{\hat{m}} (U_{j} \times R d + 1)

. Moreover, by Proposition 3.18 we have

\begin{matrix} K_{P_{1}} (x, y) = | φ_{H}^{'} (x) | K_{P_{2}} (φ (x), φ_{H}^{'} (x) y) m o d K^{\hat{m} + 1} (U_{1} \times R d + 1) . \end{matrix}

(4.2)

Therefore, if we let

K_{P_{j}, \hat{m}} \in K_{\hat{m}} (U_{j} \times R d + 1)

be the leading kernel of

K_{P_{j}}

then we get

\begin{matrix} K_{P_{1}, \hat{m}} (x, y) = | φ_{H}^{'} (x) | K_{P_{2}, \hat{m}} (φ (x), φ_{H}^{'} (x) y) . \end{matrix}

(4.3)

Next, for

j = 1, 2

we define

\begin{matrix} p_{j, m} (x, ξ) = [K_{P_{j}, \hat{m}}]_{y \to ξ}^{\land} (x, ξ), (x, ξ) \in U_{j} \times R d + 1 \ 0 . \end{matrix}

(4.4)

By Remark 3.17 for any

a \in U_{j}

the symbol

p_{j} (a, .)

yields in Heisenberg coordinates centered at

a

the principal symbol of

P_{j}

x = 0

. Moreover, since

φ_{H}^{'} (a)

is a linear map, from ( 4.3 ) we get

\begin{matrix} p_{1, m} (x, ξ) = p_{2, m} (φ (x), [φ_{H}^{'} (x)]^{- 1 t} ξ) . \end{matrix}

(4.5)

This shows that

p_{m} : = κ_{1}^{*} p_{1, m}

is an element of

S_{m} (g^{*} (V_{1} \cap V_{1}))

which is independent of the choice of the chart

κ_{1}

. Since

S_{m} (g^{*} M)

is a sheaf this gives rise to a uniquely defined symbol

p_{m} \in S_{m} (g^{*} M)

When

ℰ

is a general vector bundle, the above construction can be carried out similarly, so that we obtain:

Theorem 4.2. For any

P \in Ψ^{m} H (M, ℰ)

there is a unique symbol

σ_{m} (P) \in S_{m} (g^{*} M, ℰ)

such that if in a local trivializing Heisenberg chart

U \subset R d + 1

we let

K_{P, \hat{m}} (x, y) \in K_{\hat{m}} (U \times R d + 1)

be the leading kernel for the kernel

K_{P} (x, y)

in the form ( 3.32 ) for

P

, then we have

\begin{matrix} σ_{m} (P) (x, ξ) = [K_{P, \hat{m}}]_{y \to ξ}^{\land} (x, ξ), (x, ξ) \in U \times R d + 1 \ 0 . \end{matrix}

(4.6)

Equivalently, for any

x_{0} \in M

the symbol

σ_{m} (P) (x_{0}, .)

agrees in trivializing Heisenberg coordinates centered at

x_{0}

with the principal symbol of

P

x = 0

Definition 4.3. For

P \in Ψ^{m} H (M, ℰ)

the symbol

σ_{m} (P) \in S_{m} (g^{*} M, ℰ)

provided by Theorem 4.2 is called the principal symbol of

P

Remark 4.4. Since we have two notions of principal symbol we shall distinguish between them by saying that

σ_{m} (P)

is the global principal symbol of

P

and that in a local trivializing chart the principal symbol

p_{m}

P

in the sense of ( 3.13 ) is the local principal symbol of

P

in this chart.

In a local Heisenberg chart

U \subset R d + 1

the global symbol

σ_{m} (P)

and the local principal symbol

p_{m}

P \in Ψ^{m} H (U)

can be easily related to each other. Indeed, by Proposition 3.16 we have

\begin{matrix} p_{m} (x, ξ) = [K_{P, \hat{m}} (x, φ_{x}^{- 1} (y))]_{y \to ξ}^{\land} (x, ξ), \end{matrix}

(4.7)

where

K_{P, \hat{m}}

denotes the leading kernel for the kernel

K_{P}

in the form ( 3.32 ) for

P

. By combining this with the definition ( 4.4 ) of

σ_{m} (P)

we thus get

\begin{matrix} \begin{matrix} (4.8) & p_{m} (x, ξ) = ({\hat{φ}}_{x}^{*} σ_{m} (P)) (x, ξ), ({\hat{φ}}_{x}^{*} σ_{m} (P)) = [[σ_{m} (P)]_{ξ \to y}^{\lor} (x, φ_{x}^{- 1} (y))]_{y \to ξ}^{\land} = [φ_{x}^{*} [σ_{m} (P)]_{ξ \to y}^{\lor}]_{y \to ξ}^{\land}, \end{matrix} \end{matrix}

(4.9)

where

φ_{x}^{*}

is the isomorphism map ( 3.30 ). In particular, since the latter is a linear isomorphism of

K_{m} (U \times R d + 1)

onto itself, we see that the map

p \to {\hat{φ}}_{x}^{*} p

is a linear isomorphism of

S_{m} (U \times R d + 1)

onto itself.

Example 4.5. Let

X_{0}, \dots, X_{d}

be a local

H

-frame of

T M

near a point

a \in M

. In any Heisenberg chart associated with this frame the Heisenberg symbol of

X_{j}

\frac{1}{i} ξ_{j}

. In particular, this is true in Heisenberg coordinates centered at

a

. Thus the (global) principal symbol of

X_{j}

is equal to

\frac{1}{i} ξ_{j}

in the local trivialization of

g^{*} M \ 0

defined by the frame

X_{0}, \dots, X_{d}

. More generally, for any differential

P = \sum_{〈 α 〉 \leq m} a_{α} (x) X^{α}

M

we have

\begin{matrix} σ_{m} (P) (x, ξ) = \sum_{〈 α 〉 \leq m} a_{α} (x) i^{- | α |} ξ^{α} . \end{matrix}

(4.10)

Thus, for differential operators the global and local principal symbols agree in suitable coordinates.

Alternatively, this result follows from the fact that the isomorphism ( 3.30 ) induces the identity map on distributions

K (x, y)

supported in

U \times {y = 0}

Proposition 4.6. For every

m \in C

the principal symbol map

σ_{m} : Ψ^{m} H (M, ℰ) \to S_{m} (g^{*} M, ℰ)

gives rise to a linear isomorphism

Ψ^{m} H (M, ℰ) / Ψ^{m - 1} H (M, ℰ) \tilde{⟶} S_{m} (g^{*} M, ℰ)

Proof. By construction the principal symbol of

P \in Ψ^{m} H (M, ℰ)

depends only on his principal part in local coordinates and vanishes everywhere if, and only if, the order of

P

\leq m - 1

. Therefore, the kernel of the principal symbol map

σ_{m}

Ψ^{m - 1} H (M, ℰ)

, so

σ_{m}

induces an injective linear map

Ψ^{m} H (M, ℰ) / Ψ^{m - 1} H (M, ℰ) \to S_{m} (g^{*} M, ℰ)

To complete the proof it is enough to show that

σ_{m}

is surjective. To this end consider a symbol

p_{m} (x, ξ) \in S_{m} (g^{*} M, ℰ)

and let

(φ_{i})_{i \in I}

be a partition of the unity subordinated to an open covering

(U_{i})_{i \in I}

M

by domains of Heisenberg charts

κ_{i} : U_{i} \to V_{i}

over which there are trivializations

τ_{i} : ℰ_{|_{U_{i}}} \to U_{i} \times C^{r}

. For each index

i

let

ψ_{i} \in C^{\infty} (U_{i})

be such that

ψ_{i} = 1

near

supp φ_{i}

and set

\begin{matrix} p_{m}^{(i)} (x, ξ) = (1 - χ (ξ)) ({\hat{φ}}_{i, x}^{*} κ_{i *} τ_{i *} p_{m |_{g^{*} U_{i} \ 0}}) (x, ξ) \in S_{m} (V_{i}) \otimes End C^{r}, \end{matrix}

(4.11)

where

χ \in C^{\infty} (R d + 1)

is such that

χ = 1

near the origin and

{\hat{φ}}_{i, x}^{*}

denotes the isomorphism ( 4.9 ) with respect to the chart

V_{i}

. Then we define a a

Ψ H

DO of order

m

by letting

\begin{matrix} P = \sum φ_{i} [τ_{i}^{*} κ_{i}^{*} p_{m}^{(i)} (x, - i X)] ψ_{i} . \end{matrix}

(4.12)

For for every index

i

the local principal symbol of

φ_{i} [τ_{i}^{*} κ_{i}^{*} p_{m}^{(i)} (x, - i X)] ψ_{i}

in the chart

V_{i}

φ_{i} \circ κ_{i}^{- 1} ({\hat{φ}}_{i, x}^{*} κ_{i *} τ_{i *} p_{m |_{g^{*} U_{i} \ 0}})

, so by ( 4.8 ) its global principal is

φ_{i} \circ κ_{i}^{- 1} (κ_{i *} τ_{i *} p_{m |_{g^{*} U_{i} \ 0}})

, which pulls back to

φ_{i} p_{m}

U_{i}

. It follows that the global principal symbol of

P

σ_{m} (P) = \sum_{i} φ_{i} p_{m} = p_{m}

This proves the surjectivity of the map

σ_{m}

, so the proof is now complete. □

Next, granted the above definition of the principal symbol, we can define the model operator at a point as follows.

Definition 4.7. Let

P \in Ψ^{m} H (M, ℰ)

have (global) principal symbol

σ_{m} (P)

. Then the model operator of

P

a \in M

is the left-invariant

Ψ H

DO-operator

P^{a} : S_{0} (G_{a} M, ℰ_{a}) \to S_{0} (G_{a} M, ℰ_{x})

with symbol

σ_{m} (P)_{ξ \to y}^{\lor} (a, .)

, i.e.,

\begin{matrix} P^{a} f (x) = 〈 σ_{m} (P)_{ξ \to y}^{\lor} (a, y), f (x . y^{- 1}) 〉, f \in S_{0} (G_{a} M, ℰ_{a}) . \end{matrix}

(4.13)

Consider a local trivializing chart

U \subset R d + 1

near

a

and let us relate the model operator

P^{a}

G_{a} M

to the operator

P^{(a)} = {\tilde{p}}_{m}^{a} (- i X^{(a)})

G^{(a)}

defined using the local principal symbol

{\tilde{p}}_{m} (x, ξ)

P

in this chart. Using ( 3.6 ) and ( 4.8 ) for

f \in S_{0} (R d + 1)

we get

\begin{matrix} P^{(a)} f (y) = 〈 (p_{m}^{a})^{\lor} (z), f (y . z^{- 1}) 〉 = 〈 (σ_{m} (P)_{ξ \to y}^{\lor} (x, φ_{a}^{- 1} (y)), f (y . z^{- 1}) 〉 . \end{matrix}

(4.14)

Since

| φ_{a}^{'} | = 1

and

φ_{a}

is a Lie group isomorphism from

G^{(a)}

onto

G_{a} M

we obtain

\begin{matrix} P^{(a)} f (y) = 〈 (σ_{m} (P)_{ξ \to y}^{\lor} (x, y), f \circ φ_{a}^{- 1} (y . φ_{a} (x)^{- 1}) 〉 = (φ_{a}^{*} P^{a}) f (y) . \end{matrix}

(4.15)

In particular, we have

\begin{matrix} P^{a} = (φ_{a})_{*} p_{m}^{a} (- i X^{(a)}) . \end{matrix}

(4.16)

4.2 Composition of principal symbols and model operators

Let us now look at the composition of principal symbols. To this end for

a \in M

we let

*^{a} : S_{m_{1}} (R d + 1) \times S_{m_{2}} (R d + 1) \to S_{m_{1} + m_{2}} (R d + 1)

be the convolution product for symbols defined by the product law of

G_{a} M

under the identification

G_{a} M ≃ R d + 1

provided by a

H

-frame

X_{0}, \dots, X_{d}

T M

near

a

, that is,

\begin{matrix} (p_{m_{1}} *^{a} p_{m_{j}}) (- i X^{a}) = p_{m_{1}} (- i X^{a}) \circ p_{m_{2}} (- i X^{a}), p_{m_{j}} \in S_{m_{j}} (R d + 1) . \end{matrix}

(4.17)

Let

U \subset R d + 1

be a local trivializing Heisenberg chart chart near

a

and for

j = 1, 2

let

P_{j} \in Ψ^{m_{j}} H (U)

have (global) principal symbol

σ_{m_{j}} (P_{j})

. Recall that under the trivialization of

G U

provided by the

H

-frame

X_{0}, \dots, X_{d}

we have

P_{j}^{a} = σ (P_{j}) (x, - i X^{a})

. Thus,

\begin{matrix} [σ_{m_{j}} (P_{j}) (x,,) *^{a} σ_{m_{j}} (P_{j}) (x, .)] (- i X^{a}) = P_{1}^{a} P_{2}^{a} . \end{matrix}

(4.18)

On the other hand, using ( 4.8 ) and ( 4.16 ) we see that

{\hat{φ}}_{a}^{*} [p_{m_{1}} *^{a} p_{m_{2}}] (- i X^{a})

is equal to

(4.19) φ a * [ p m 1 ( − i X a ) ∘ p m 2 ( − i X a ) ] = φ a * [ p m 1 ( − i X a ) ] ∘ φ a * [ p m 2 ( − i X a ) ] = ( φ ^ a * p m 1 ) ( − i X ( a ) ) ∘ ( φ ^ a * p m 2 ) ( − i X ( a ) ) = [ ( φ ^ a * p m 1 ) * ( a ) ( φ ^ a * p m 2 ) ] ( − i X ( a ) ) .

Hence we have

\begin{matrix} p_{m_{1}} *^{a} p_{m_{2}} = ({\hat{φ}}_{a})_{*} [({\hat{φ}}_{a}^{*} p_{m_{1}}) *^{(a)} ({\hat{φ}}_{a}^{*} p_{m_{2}})] \forall p_{m_{j}} \in S_{m_{j}} (R d + 1), \end{matrix}

(4.20)

where

({\hat{φ}}_{a})_{*}

denotes the inverse of

{\hat{φ}}_{a}^{*}

. Since

{\hat{φ}}_{a}^{*}

, its inverse and

*^{(a)}

depend smoothly on

a

, we deduce that that so does

*^{a}

. Therefore, we obtain:

Proposition 4.8. The group laws on the fibers of

G M

give rise to a convolution product,

\begin{matrix} \begin{matrix} (4.21) & * : S_{m_{1}} (g^{*} M, ℰ) \times S_{m_{2}} (g^{*} M, ℰ) ⟶ S_{m_{1} + m_{2}} (g^{*} M, ℰ), p_{m_{1}} * p_{m_{2}} (x, ξ) = [p_{m_{1}} (x, .) *^{x} p_{m_{2}} (x, .)] (ξ), p_{m_{j}} \in S_{m_{j}} (g^{*} M, ℰ), \end{matrix} \end{matrix}

(4.22)

where

*^{x}

denote the convolution product for symbols on

G_{x} M

Notice that ( 4.20 ) shows that, under the relation ( 4.8 ) between local and global principal symbols, the convolution product ( 4.17 ) for global principal symbols corresponds to the convolution product ( 3.19 ) for local principal symbols. Since by Proposition 3.9 the latter yields the local principal symbol of the product of two

Ψ H

DO's in a local chart, we deduce that the convolution product ( 4.17 ) yields the global principal symbol of the product two

Ψ H

DO's. Moreover, by ( 4.18 ) the global convolution product ( 4.17 ) corresponds to the product of model operators, so the model operator of a product of two

Ψ H

DO's is equal to the product of the model operators. We have thus proved:

Proposition 4.9. For

j = 1, 2

let

P_{j} \in Ψ^{m_{j}} H (M, ℰ)

and suppose that

P_{1}

P_{2}

is properly supported.

1) We have

σ_{m_{1} + m_{2}} (P_{1} P_{2}) = σ_{m_{1}} (P) * σ_{m_{2}} (P)

2) At every

a \in M

the model operator of

P_{1} P_{2}

(P_{1} P_{2})^{a} = P_{1}^{a} P_{2}^{a}

4.3 Principal symbol of transposes and adjoints

In this subsection we shall determine the principal symbols and the model operators of transposes and adjoints of

Ψ H

DO's. Recall that if

P \in Ψ^{m} H (M, ℰ)

then by Proposition 3.23 its transpose

P^{t} : C_{c}^{\infty} (M, ℰ^{*}) \to C^{\infty} (M, ℰ)

is a

Ψ H

DO of order

m

and its adjoint

P^{*} : C_{c}^{\infty} (M, ℰ) \to C^{\infty} (M, ℰ)

is a

Ψ H

DO of order

\bar{m}

(assuming

M

endowed with a positive density and

ℰ

with a Hermitian metric in order to define the adjoint).

Proposition 4.10. Let

P \in Ψ^{m} H (M, ℰ)

have principal symbol

σ_{m} (P)

. Then: 1) The principal symbol of

P^{t}

σ_{m} (P^{t}) (x, ξ) = σ_{m} (x, - ξ)^{t}

(this is an element of

S_{m} (g^{*} M, ℰ *)

); 2) If

P^{a}

is the model operator of

P

a

then the model operator of

P^{t}

a

is the transpose operator

(P^{a})^{t} : S_{0} (G_{x} M, ℰ_{x}^{*}) \to S_{0} (G_{x} M, ℰ_{x}^{*})

Proof. Let us first assume that

ℰ

is the trivial line bundle and that

P

is a scalar operator. In a local Heisenberg chart

U \subset R d + 1

we can write the distribution kernels of

P

and

P^{t}

in the form ( 3.32 ) with

K_{P}

and

K_{P^{t}}

K^{\hat{m}} (U \times R d + 1)

. Let

K_{P, \hat{m}}

and

K_{P^{t}, \hat{m}}^{t}

denote the principal parts of

K_{P}

and

K_{P^{t}}

respectively. Then the principal symbols of

P

and

P^{t}

are

σ_{m} (P) (x, ξ) = (K_{P, \hat{m}})_{y \to ξ}^{\land} (x, ξ)

and

σ_{m} (P^{t}) (x, ξ) = (K_{P^{t}, \hat{m}})_{y \to ξ}^{\land} (x, ξ)

respectively. Since ( 3.38 ) implies that

K_{P^{t}, \hat{m}} (x, y) = K_{P, \hat{m}} (x, - y)

and the Fourier transform commutes with the multiplication by

- 1

we get

\begin{matrix} σ_{m} (P^{t}) (x, ξ) = σ_{m} (P) (x, - ξ) . \end{matrix}

(4.23)

Next, for

a \in U

let

p \in S_{m} (G_{a})

and let

P

be the left-invariant

Ψ H

DO with symbol

p

. Then the transpose

P^{t}

is such that, for

f

and

g

S_{0} (G_{a} U)

, we have

(4.24) 〈 P t f , g 〉 = 〈 f , P v 〉 = 〈 1 , f ( x ) ( P g ) ( x ) 〉 = 〈 1 , f ( x ) 〈 p ˇ ( y ) , g ( x . y − 1 ) 〉 〉 = 〈 1 ⊗ p ˇ ( x , y ) , f ( x ) g ( x . y − 1 ) 〉 .

Therefore, using the change of variable

(x, y) \to (x . y^{- 1}, y^{- 1})

and the fact

y^{- 1} = - y

we get

\begin{matrix} 〈 P^{t} f, g 〉 = 〈 1 \otimes \overset{ˇ}{p} (x, - y), f (x) g (x . y^{- 1}) 〉 = 〈 1, f (x) 〈 \overset{ˇ}{p} (- y), g (x . y^{- 1}) 〉 〉 . \end{matrix}

(4.25)

Since

\overset{ˇ}{p} (- y) = {\overset{ˇ}{p}}^{t} (y)

with

p^{t} (ξ) = p (- ξ)

, we obtain

\begin{matrix} 〈 P^{t} f, g 〉 = 〈 1, f (x) 〈 {\overset{ˇ}{p}}^{t} (y), g (x . y^{- 1}) 〉 〉 = 〈 (p^{t} * f) (x), g (x) 〉 . \end{matrix}

(4.26)

Thus

P^{t}

is the left-convolution operator with symbol

p^{t} (ξ) = p (- ξ)

Now, since the model operator

(P^{t})^{a}

is the left-invariant

Ψ H

DO with symbol

σ_{m} (P^{t}) (a, ξ) = σ_{m} (P) (a, - ξ)

, it follows that

(P^{t})^{a}

agrees with the transpose of

P^{a}

In the general case, when

ℰ

is not the trivial bundle, we can similarly show that

P^{t}

is a

Ψ H

DO of order

m

with principal symbol

σ_{m} (P^{t}) (x, ξ) = σ_{m} (P) (x, - ξ)^{t}

and such that at every point

a \in M

its model operator at

a

is the transpose

(P^{a})^{t}

P^{a}

. □

Assume now that

M

is endowed with a positive density and

ℰ

with a Hermitian metric respectively and let

L^{2} (M, ℰ)

be the associated

L^{2}

-Hilbert space.

Proposition 4.11. Let

P \in Ψ^{m} H (M, ℰ)

have principal symbol

σ_{m} (P)

. Then: 1) The principal symbol of

P^{*}

σ_{\bar{m}} (P^{*}) (x, ξ) = σ_{m} (P) (x, ξ)^{*}

2) If

P^{x}

denotes the model operator of

P

x \in M

then the model operator of

P^{*}

x

is the adjoint

(P^{x})^{*}

P^{x}

Proof. Let us first assume that

ℰ

is the trivial line bundle, so that

P

is a scalar operator. Moreover, since the above statements are local ones, it is enough to prove them in a local Heisenberg chart

U \subset R d + 1

and we may assume that

P

is a

Ψ H

DO on

U

Let

\bar{P} : C_{c}^{\infty} (U) \to C^{\infty} (U)

be the conjugate operator of

P

, so that

\bar{P} u = \bar{P (\bar{f})}

for any

f \in C_{c}^{\infty} (U)

By Proposition 3.16 the distribution kernel of

P

of the form ( 3.32 ) with

K_{P} (x, y)

K_{\hat{m}} (U \times R d + 1)

, so the kernel of

\bar{P}

takes the form

\begin{matrix} k_{\bar{P}} (x, y) = \bar{k_{P} (x, y)} = | ɛ_{x}^{'} | \bar{K_{P} (x, y)} m o d C^{\infty} (U \times U) . \end{matrix}

(4.27)

Since the conjugation of distribution

K (x, y) \to \bar{K (x, y)}

induces an anti-linear isomorphism from

K^{\hat{m}} (U \times R d + 1)

onto

K^{\hat{\bar{m}}} (U \times R d + 1)

, it follows from Proposition 3.16 that

\bar{P}

is a

Ψ H

DO of order

\hat{m}

and its kernel can be put into the form ( 3.32 ) with

K_{\bar{P}} (x, y) = \bar{K_{P} (x, y)}

. In particular, if we let

K_{P, \hat{m}} \in K_{\hat{m}} (U \times R d + 1)

denote the leading kernel of

K_{P}

then the leading kernel of

K_{\bar{P}}

\bar{K_{P, \hat{m}}}

Thus

\bar{P}

has principal symbol,

\begin{matrix} σ_{\bar{m}} (\bar{P}) (x, ξ) = [\bar{K_{P, \hat{m}}}]_{ξ \to y}^{\land} (x, ξ) = \bar{[(K_{P, \hat{m}})_{ξ \to y}^{\land} (x, - ξ)} = \bar{σ_{m} (x, - ξ)} . \end{matrix}

(4.28)

Moreover, since

σ_{\bar{m}} (\bar{P})_{ξ \to y}^{\lor} (x, y) = \bar{σ_{m} (P)_{ξ \to y}^{\lor} (x, y)}

the model operator at

a \in U

\bar{P}

is such that, for any

f \in S_{0} (G_{a} U)

, we have

\begin{matrix} (\bar{P})^{a} f (x) = 〈 \bar{σ_{m} (P)_{ξ \to y}^{\lor} (x, y)}, f (x . y^{- 1}) 〉 = \bar{〈 σ_{m} (P)_{ξ \to y}^{\lor} (x, y), \bar{f (x . y^{- 1}} 〉} = \bar{P^{a}} f (x), \end{matrix}

(4.29)

(\bar{P})^{a}

agrees with

\bar{P^{a}}

Combining this with Proposition 3.21 and Proposition 4.10 we thus see that

{\bar{P}}^{t}

is a

Ψ H

DO of order

\bar{m}

such that:

If we put the kernel of

{\bar{P}}^{t}

into the form ( 3.32 ) with respect to

K_{{\bar{P}}^{t}} (x, y) \in K_{\hat{\bar{m}}} (U \times R d + 1)

, then the leading kernel of

K_{{\bar{P}}^{t}}

K_{{\bar{P}}^{t}, \hat{\bar{m}}} = \bar{K_{P, \hat{m}} (x, - y)}

; The global principal symbol of

{\bar{P}}^{t}

σ_{\bar{m}} ({\bar{P}}^{t}) = \bar{σ_{m} (P^{t}) (x, ξ)} = \bar{σ_{\bar{m}} (P) (x, ξ)}

; The model operator at

a \in U

{\bar{P}}^{t}

({\bar{P}}^{t})^{a} = {\bar{P^{a}}}^{t} = (P^{a})^{*}

Now, let

d ρ (x) = ρ (x) d x

be the smooth positive density on

U

coming from that of

M

. The formal adjoint

P^{*} : C_{c}^{\infty} (U) \to C^{\infty} (U)

P

with respect to

d ρ

is such that

\begin{matrix} \int_{f} \bar{P f (x)} g (x) ρ (x) d x = \int_{U} \bar{f (x)} P^{*} g (x) ρ (x) d x, f, g \in C_{c}^{\infty} (U) . \end{matrix}

(4.30)

Thus

P^{*} = ρ^{- 1} {\bar{P}}^{t} ρ

, which shows that

P^{*}

is a

Ψ H

DO of order

\bar{m}

. Moreover, as in the proof of Proposition 3.21 in Section 9 , we can prove that the kernel of

P^{*}

can be put into the form ( 3.32 ) with

K_{P^{*}} (x, y) \in K^{\bar{m}} (U \times R d + 1)

such that

\begin{matrix} K_{P^{*}} (x, y) = ρ (x)^{- 1} K_{{\bar{P}}^{t}} (x, y) ρ (ɛ_{x}^{- 1} (y)) \sim \sum_{α} \frac{1}{α!} ρ (x)^{- 1} \partial_{y} (ρ (ɛ_{x}^{- 1} (y))_{|_{y = 0}} K_{{\bar{P}}^{t}} (x, y) . \end{matrix}

(4.31)

In particular, the kernels

K_{P^{*}} (x, y)

and

K_{{\bar{P}}^{t}} (x, y)

agree modulo

K^{\hat{\bar{m}} + 1} (U \times R d + 1)

, hence have same leading kernel. It then follows that

P^{*}

and

{\bar{P}}^{t}

have same principal symbol and same model operator at a point

a \in U

, that is, we have

σ_{\bar{m}} (P^{*}) (a, ξ) = \bar{σ_{m} (P) (x, ξ)}

and

(P^{*})^{a} = (P^{a})^{*}

Finally, assume that

ℰ

is a general bundle, so that the restriction of

P

U

is given by a matrix

P = (P_{i j})

Ψ H

DO's of order

r

. Let

h (x) \in C^{\infty} (U, G L_{r} (C))

h (x)^{*} = h (x)

, be the Hermitian metric on

U \times C^{r}

coming from that of

ℰ

. Then the adjoint of

P

with respect to this Hermitian metric is

P^{*} = ρ^{- 1} h^{- 1} {\bar{P}}^{t} h ρ

. Therefore, in a the same way as in the scalar case we can prove that

P^{*}

has principal symbol

h (x)^{- 1} {\bar{σ_{m} (P) (x, ξ)}}^{t} h (x) = σ_{m} (P) (x, ξ)^{*}

and its model operator at any point

a \in U

h (a)^{- 1} {\bar{P^{a}}}^{t} h (a) = (P^{a})^{*}

. □

5 Hypoellipticity, parametrices and the Rockland condition

In this section we define a Rockland condition for

Ψ H

DO's and relate it to the invertibility of the principal symbol to get hypoellipticity criterions.

First, by [BG,Sect. 18] in a local Heisenberg chart the invertibility of the local principal symbol of a

Ψ H

DO is equivalent to the existence of a

Ψ H

DO-parametrix. Using the global principal symbol we can give a global reformulation of this result as follows.

Proposition 5.1. Let

P : C_{c}^{\infty} (M, ℰ) \to C^{\infty} (M, ℰ)

be a

Ψ H

DO of order

m

such that

k : = ℜ m > 0

. The following are equivalent:

1) The principal symbol

σ_{m} (P)

P

is invertible with respect to the convolution product for homogeneous symbols; 2)

P

admits a parametrix

Q

Ψ^{- m} H (M, ℰ)

, so that

P Q = Q P = 1 m o d Ψ - \infty (M, ℰ)

Proof. First, it immediately follows from Proposition 4.9 that 2) implies 1). Conversely, in a local trivializing Heisenberg chart ( 4.20 ) shows that the invertibility of the global principal $σ_{m} (P)$ is equivalent to that of the local principal symbol. Once the latter is granted Lemma 3.7 and Proposition 3.9 allows us to carry out in a local trivializing Heisenberg chart the standard parametrix construction in a trivializing Heisenberg chart to get a parametrix for $P$ as a $Ψ H$ DO of order $- m$ (see [BG,p. 142] ). A classical partition of the unity argument then allows us to produce a parametrix for $P$ in $Ψ^{- m} H (M, ℰ)$ . □

When a

Ψ H

DO has an invertible principal symbol using the Sobolev regularity properties of its parametrices allows us to get:

Proposition 5.2 ([BG,p. 142] ). Let

P : C_{c}^{\infty} (M, ℰ) \to C^{\infty} (M, ℰ)

be a

Ψ H

DO of order

m

whose principal symbol is invertible. If

k : = ℜ m > 0

then

P

is hypoelliptic with loss of

\frac{k}{2}

-derivatives, i.e., for any

a \in M

, any

u \in ℰ^{'} (M, ℰ)

and any

s \in R

, we have

\begin{matrix} P u is L_{s}^{2} near a ⟹ u is L_{s + k}^{2} near a . \end{matrix}

(5.1)

In particular, if

M

is compact then, for any reals

s

and

s^{'}

, we have the estimate,

\begin{matrix} ∥ f ∥_{L_{s + k}^{2}} \leq C_{s s^{'}} (∥ P f ∥_{L_{s}^{2}} + ∥ f ∥_{L_{s^{'}}^{2}}), u \in C^{\infty} (M, ℰ) . \end{matrix}

(5.2)

Remark 5.3. We can give sharper hypoellipticity regularity results for

Ψ H

DO's in terms of suitably weighted Sobolev spaces (see [FS] , [Po3] ). When

P

is a differential operator and the Levi form is non-vanishing these results correspond to the maximal hypoellipticity of

P

as in [HN3] .

Now, let

P : C^{\infty} (M, ℰ) \to C^{\infty} (M, ℰ)

be a

Ψ

DO of order

m

and assume that

M

is endowed with a positive density and

ℰ

with a Hermitian metric. Let

P^{a}

be the model operator of

P

at a point

a \in M

and let

π : G \to ℋ_{π}

be a (nontrivial) unitary representation of

G = G_{a} M

. We define the symbol

π_{P^{a}}

as follows (see also [Ro1] , [Gł] , [CGGP] ).

Let

ℋ_{π}^{0} (ℰ_{a})

be the subspace of

ℋ_{π} (ℰ_{a}) : = ℋ_{π} \otimes ℰ_{a}

spanned by the vectors of the form

\begin{matrix} π_{f} ξ = \int_{G} (π_{x} \otimes 1_{ℰ_{a}}) (ξ \otimes f (x)) d x, \end{matrix}

(5.3)

ξ

ranges over

ℋ_{π}

and

f

over

S_{0} (G, ℰ_{a}) = S_{0} (G) \otimes ℰ_{a}

. Then we let

π_{P^{a}}

denote the (unbounded) operator of

ℋ_{π} (ℰ_{a})

with domain

ℋ_{π}^{0} (ℰ_{a})

such that

\begin{matrix} π_{P^{a}} (π_{f} ξ) = π_{P^{a} f} ξ \forall f \in S_{0} (G, ℰ_{a}) \forall ξ \in ℋ_{π} . \end{matrix}

(5.4)

One can check that

π_{P^{a *}}

is the adjoint of

π_{P^{a}}

ℋ_{π}^{0}

, hence is densely defined. Thus

π_{P^{a}}

is closeable and we can let

\bar{π_{P^{a}}}

denotes its closure. In the sequel we let

C_{π}^{\infty} (ℰ_{a}) = C_{π}^{\infty} \otimes ℰ_{a}

, where

C_{π}^{\infty} \subset ℋ_{π}

denotes the space of smooth vectors of

π

(i.e. the subspace of vectors

ξ \in ℋ_{π}

so that

x \to π (x) ξ

is smooth from

G

ℋ_{π}

Proposition 5.4 ([CGGP] ). 1) The domain of

\bar{π_{P^{a}}}

always contains

C_{π}^{\infty} (ℰ_{a})

2) If

ℜ m \leq 0

then the operator

\bar{π_{P^{a}}}

is bounded.

3) We have

\bar{(π_{P^{a}})^{*}} = (\bar{π_{P^{a}}})^{*}

4) If

P_{1}

and

P_{2}

are

Ψ

DO's on

M

then

\bar{π_{(P_{1} P_{2})^{a}}} = \bar{π_{P_{1}^{a}}} \bar{π_{P_{2}^{a}}}

Remark 5.5. If

ℰ_{a} = C

and

P^{a}

is a differentiable operator then, as it is left-invariant,

P^{a}

belongs to the enveloping algebra

U (g)

of the Lie algebra

g = g_{a} M

G

. In this case

\bar{π_{P^{a}}}

coincides on

C_{π}^{\infty}

with the operator

d π (P^{a})

, where

d π

is the representation of

U (g)

induced by

π

Definition 5.6. We say that

P

satisfies the Rockland condition at

a

if for any nontrivial unitary irreducible representation

π

G_{a} M

the operator

\bar{π_{P^{a}}}

is injective on

C_{π}^{\infty} (ℰ_{a})

Since

G = G_{a} M ≃ H^{2 n + 1} \times R^{d - 2 n}

with

2 n = rk ℒ_{a}

, there are left-invariant vector fields

X_{0}, \dots, X_{d}

G

such that

X_{0}, \dots, X_{2 n}

are the left-invariant vector fields on

H^{2 n + 1}

given by ( 2.2 ) and

X_{k} = \frac{\partial}{\partial x_{k}}

for

k = 2 n + 1, \dots, d

. Then, up to unitary equivalence, the nontrivial irreducible representations of

G

are of two types:

(i) Infinite dimensional representations

π^{λ, ξ} : G \to L^{2} (R^{n})

parametrized by

λ \in R \ 0

and

ξ = (ξ_{2 n + 1}, \dots, ξ_{2 n})

such that

\begin{matrix} \begin{matrix} (5.5) & d π^{λ, ξ} (X_{0}) = i λ | λ |, d π^{λ, ξ} (X_{k}) = i λ ξ_{k}, k = 2 n + 1, \dots, d, d π^{λ, ξ} (X_{j}) = | λ | \frac{\partial}{\partial ξ_{j}}, d π^{λ, ξ} (X_{n + j}) = i λ ξ_{j}, j = 1, \dots, n . \end{matrix} \end{matrix}

(5.6)

Moreover, in this case we have

C^{\infty} (π^{\pm, ξ}) = S (R^{n})

(ii) One dimensional representations

π^{ξ} : G \to C

indexed by

ξ = (ξ_{1}, \dots, ξ_{d}) \in R^{d} \ 0

such that

\begin{matrix} d π^{ξ} (X_{0}) = 0, d π^{ξ} (X_{j}) = i ξ_{j}, j = 1, \dots, d . \end{matrix}

(5.7)

In particular, if

P = p_{m} (- i X)

with

p \in S_{m} (G)

then the homogeneity of the symbol

p

implies that we have

π_{P}^{λ, ξ} = | λ |^{m} π_{P}^{\pm, ξ}

where

\bar{π_{P}^{\pm, ξ}} = \bar{π_{P}^{\pm 1, ξ}}

accordingly with the sign of

λ

On the other hand, for the representations in (ii) we have

\bar{π_{P}^{ξ}} = π_{P}^{ξ} = p_{m} (0, ξ_{1}, \dots, ξ_{d})

. Therefore, we get:

Proposition 5.7. The Rockland condition for

P = p_{m} (- i X^{a})

p_{m} \in S_{m} (G_{a} M)

, holds, and only if, the following two conditions are satisfied:

(i) The operators

\bar{π_{P}^{\pm, ξ}}

ξ \in R^{d - 2 n}

, are injective on

S (R^{n})

; (ii) The restriction of the symbol

p_{m}

{0} \times (R^{n} \ 0) ≃ H_{a}^{*} \ 0

is pointwise invertible.

Remark 5.8. In the case of

G_{a} M = H^{2 n + 1}

the conditions (i) and (ii) have those considered by Taylor [Tay] .

Next, if

P \in Ψ^{m} H (M, ℰ)

has an invertible principal symbol, hence admits a parametrix

Q \in Ψ^{- m} H (M, ℰ)

, then for any

a \in M

we have

Q^{a} P^{a}

and

(Q^{a})^{t} (P^{a})^{t}

are equal to

1

S_{0} (G_{a} M, ℰ_{a})

and

S_{0} (G_{a} M, ℰ_{a}^{*})

respectively. It then follows from Proposition 5.4 that for any nontrivial irreducible unitary representation

π

G_{a}

the operators

\bar{π_{P^{a}}}

and

\bar{π_{(P^{a})^{t}}}

are injective on

C^{\infty} (π)

. Thus

P

and

P^{t}

satisfy the Rockland condition at every point of

M

. Conversely, if at some point

a \in M

the model operator

(P^{a})^{t}

satisfies the Rockland condition then

(P^{a})^{t}

is hypoelliptic (see [Ro1] , [Bea] , [Gł] ; see also [HN1] , [HN2] , [CGGP] ). The latter fact then implies that

P^{a}

admits a fundamental solution

k^{a} \in S^{'} (G_{a} M, ℰ_{a})

so that

P^{a} k^{a} = δ_{a}

(see [Fo] , [Ge] , [CGGP] ). In particular, the inverse Fourier transform of

k^{a}

yields an inverse for the symbol of

P^{a}

This shows that if

P

and

P^{t}

satisfies the Rockland condition at every point then for any

a \in M

there exists

q_{- m}^{a} \in S_{- m} (g_{a}^{*} M, ℰ_{a})

so that

q^{a} *^{a} σ_{m} (P) (a, .) = σ_{m} (P) (a, .) *^{a} q^{a} = 1

. However, it is an open problem to determine whether

q^{a}

depends smoothly

a

and so define an element of

S_{m} (g^{*} M, ℰ)

The above issue is at least true in the case of sublaplacians (see [BG] and next section). As we shall now see this result can be extended to arbitrary

Ψ H

DO's when the Levi form of

(M, H)

has constant rank. More precisely, we have:

Proposition 5.9. Suppose that the Levi form of

(M, H)

has constant rank and let

P \in Ψ^{m} H (M, ℰ)

. Then the following are equivalent:

(i)

P

and

P^{t}

satisfy the Rockland condition at every point of

M

; (ii)

P

and

P^{*}

satisfy the Rockland condition at every point of

M

; (iii) The principal symbol of

P

is invertible.

In particular, if (i) holds then

P

admits a parametrix in

Ψ^{- m} H (M, ℰ)

and is hypoelliptic with loss of

\frac{1}{2} ℜ m

derivatives.

Proof. We saw that (i) implies (iii) already. The same argument shows that (ii) implies (i). Thus we only need to prove the converse statements.

Assume that

P

and

P^{t}

satisfies the Rockland condition at every point. For sake of simplicity let us further assume that

ℰ

is the trivial line bundle, so that

P

and

P^{t}

are scalar operators. We need to show that the principal symbol of

P

is invertible. As this is a local issue it is enough to prove it in a local Heisenberg chart

U \subset R d + 1

, so we may assume that

P

and

P^{t}

are

Ψ H

DO's on

U

Moreover, since the Levi form has constant rank it follows from [Po2,Prop. 2.8] that

G M

is a fiber bundle of Lie group with fiber

G = H^{2 n + 1} \times R^{d - 2 n}

, where

2 n

is the rank of the Levi form.

Therefore, by considering a trivialization of this fiber bundle by means of a suitable local

H

-frame (see [Po2,p. 5] ) we may further assume that

G U

is the trivial bundle

U \times G

. In particular, the families of model operators

(P^{x})_{x \in U}

and

((P^{t})^{x})_{x \in U}

can be seen as smooth families of left-invariant

Ψ H

DO's on

G

as in [CGGP] .

Now, since

P^{x}

satisfies the Rockland condition for every

x \in U

it follows from [CGGP,Thm. 5(d)] near every

x_{0} \in U

there exists an open neighborhood

V \subset U

x_{0}

and a smooth family

(K^{x})_{x \in V} \subset K_{m - d - 2} (G)

such that if for

x \in V

we let

Q^{x}

be the left-convolution operator with

K^{x}

acting on

S_{0} (G)

then

Q^{x}

is a left-inverse for

P^{x}

(P^{t})^{x} = (P^{x})^{t}

satisfies the Rockland condition for every

x \in V

the same arguments show that

(P^{x})^{t}

is left-invertible on

S_{0} (G)

for any

x

in an open neighborhood

W

x_{0}

contained in

V

Thus

P^{x}

is invertible with two-side inverse

Q^{x}

Since

Q^{x}

is the left-invariant

Ψ H

DO with symbol

q_{- m}^{x} = (K^{x})^{\land}

S_{- m} (G)

we see that we have

\begin{matrix} q_{- m}^{x} * σ_{m} (P) (x, .) = σ_{m} (P) (x, .) * q_{- m}^{x} = 1 \forall x \in W . \end{matrix}

(5.8)

In fact, as

K^{x}

depends smoothly on

x

, that is, yields an element of

K_{- m} (W \times G)

, we get a symbol in

S_{- m} (W \times g^{*})

by letting

q_{- m} (x, ξ) = q_{- m}^{x} (ξ)

. Then ( 5.8 ) shows that

q_{- m}

is an inverse for

σ_{m} (P)

W \times (g^{*} \ 0)

. This shows that

σ_{m} (P)

is invertible near every point of

U

, so

σ_{m} (P)

is an invertible symbol.

Next, since the aforementioned result of [CGGP] remains valid mutatis standis for systems, by working in a local trivializing Heisenberg chart we can similarly show that, in the case

ℰ

is a general bundle, if

P

and

P^{t}

satisfy the Rockland condition at every point then the the principal symbol of

P

is invertible. Thus the assertions (i) and (ii) are equivalent. Finally, the above arguments remain valid when we replace the transpose

P^{t}

by the adjoint of

P

, so the statements (ii) and (iii) are equivalent. □

In particular, when the Levi form of

(M, H)

has constant rank and

P

is selfadjoint the Rockland condition for

P

is equivalent to the invertibility of the principal symbol of

P

, hence is equivalent to the existence of a parametrix in

Ψ^{- m} H (M, ℰ)

6 Hypoellipticity criterions for sublaplacians

The main focus of this section is on sublaplacians, which furnish several important examples of operators on Heisenberg manifolds. The scalar case was dealt with in [BG] , but the results were not extended to sublaplacians acting on sections of vector bundles. These extensions are necessary in order to deal with sublaplacians acting on forms such as the Kohn Laplacian or the horizontal sublaplacian (see next section).

In this section, after having explained the scalar case from the point of view this paper, we extend the results to the non-scalar case. In particular, this will allow us to complete the treatment of the Kohn Laplacian in [BG] (see Remark 7.3 ahead).

Definition 6.1. A differential operator

Δ : C^{\infty} (M, ℰ) \to C^{\infty} (M, ℰ)

is a sublaplacian when, near any point

a \in M

, we can put

Δ

in the form,

\begin{matrix} Δ = - (X_{1}^{2} + \dots + X_{d}^{2}) - i μ (x) X_{0} + O_{H} (1), \end{matrix}

(6.1)

where

X_{0}, X_{1}, \dots, X_{d}

is a local

H

-frame of

T M

, the coefficient

μ (x)

is a local section of

End ℰ

and the notation

O_{H} (1)

means a differential operator of Heisenberg order

\leq 1

Let us look at the Rockland condition for a scalar sublaplacian

Δ : C^{\infty} (M) \to C^{\infty} (M)

. Let

a \in M

and let

X_{0}, X_{1}, \dots, X_{d}

be a local

H

-frame of

T M

so that near

a

we can write

\begin{matrix} Δ = - \sum_{j = 1}^{d} X_{j}^{2} - i μ (x) X_{0} + O_{H} (1), \end{matrix}

(6.2)

where

μ (x)

is a smooth function near

a

. Using ( 4.10 ) we see that the principal symbol of

Δ

\begin{matrix} σ_{2} (Δ) (x, ξ) = | ξ^{'} |^{2} + μ (a) ξ_{0}, ξ^{'} = (ξ_{1}, \dots, ξ_{d}) . \end{matrix}

(6.3)

In particular we have

σ_{2} (Δ) (x, 0, ξ^{'}) = | ξ^{'} |^{2} > 0

for

ξ^{'} \neq 0

, which shows that the condition (i) of Proposition 5.7 is always satisfied.

Let

L (x) = (L_{j k} (x))

be the matrix of the Levi form

ℒ

with respect to the

H

-frame

X_{0}, \dots, X_{d}

, so that for

j, k = 1, \dots, d

we have

\begin{matrix} ℒ (X_{j}, X_{k}) = [X_{j}, X_{k}] = L_{j k} X_{0} m o d H . \end{matrix}

(6.4)

Equivalently, if we let

g (x)

be the metric on

H

making orthonormal the frame

X_{1}, \dots, X_{d}

, then for any sections

X

and

Y

H

we have

\begin{matrix} ℒ (X, Y) = g (x) (L (x) X, Y) X_{0} m o d H . \end{matrix}

(6.5)

The matrix

L (x)

is antisymmetric, so up to an orthogonal change of frame of

H

, which does not affect the form ( 6.2 ), we may assume that

L (a)

is in the normal form,

\begin{matrix} L (a) = (\begin{matrix} 0 & D & 0 \\ - D & 0 & 0 \\ 0 & 0 & 0 \end{matrix}), D = diag (λ_{1}, \dots, λ_{n}), λ_{j} > 0, \end{matrix}

(6.6)

so that

\pm i λ_{1}, \dots, \pm λ_{2 n}, 0, \dots, 0

are the eigenvalues of

L (a)

counted with multiplicity, the multiplicity of

0

being

d - 2 n

. Then the model vector fields

X_{0}^{a}, \dots, X_{d}^{a}

are:

\begin{matrix} \begin{matrix} (6.7) & X_{0}^{a} = \frac{\partial}{\partial x_{0}}, X_{k}^{a} = \frac{\partial}{\partial x_{k}}, k = 2 n + 1, . ., d, X_{j}^{a} = \frac{\partial}{\partial x_{j}} - \frac{1}{2} λ_{j} x_{n + j} \frac{\partial}{\partial x_{0}}, X_{n + j}^{a} = \frac{\partial}{\partial x_{j}} + \frac{1}{2} λ_{j} x_{j} \frac{\partial}{\partial x_{0}}, j = 1, \dots, n . \end{matrix} \end{matrix}

(6.8)

In terms of these vector fields the model operator of

Δ

a

\begin{matrix} Δ^{a} = - [(X_{1}^{a})^{2} + \dots + (X_{1}^{a})^{2}] - i μ (a) X_{0}^{a} . \end{matrix}

(6.9)

Next, under the isomorphism

φ : H^{2 n + 1} \times R^{d - 2 n} \to G_{a} M

given by

\begin{matrix} φ (x_{0}, \dots, x_{d}) = (x_{0}, λ_{1}^{\frac{1}{2}} x_{1}, \dots, λ_{n}^{\frac{1}{2}} x_{n}, λ_{1}^{\frac{1}{2}} x_{n + 1}, \dots λ_{n}^{\frac{1}{2}} x_{2 n}, x_{2 n + 1}, \dots, x_{d}), \end{matrix}

(6.10)

the representations

π^{\pm, ξ} = π^{\pm 1, ξ}

ξ \in {0}^{2 n} \times R^{d - 2 n}

, become the representations of

G_{a} M

such that

\begin{matrix} \begin{matrix} (6.12) & \begin{matrix} (6.11) & d π^{\pm, ξ} (X_{0}) = \pm i, d π^{\pm, ξ} (X_{k}) = \pm i ξ_{k}, k = 2 n + 1, \dots, d, d π^{\pm, ξ} (X_{j}) = λ_{j}^{\frac{1}{2}} \frac{\partial}{\partial ξ_{j}}, d π^{\pm, ξ} (X_{n + j}) = \pm i λ_{j}^{\frac{1}{2}} ξ_{j}, j = 1, \dots, n, π_{Δ^{a}}^{\pm, ξ} = d π^{\pm, ξ} (Δ^{a}) = \sum_{j = 1}^{n} λ_{j} (- \partial_{ξ_{j}}^{2} + ξ_{j}^{2}) \pm (ξ_{2 n + 1}^{2} + \dots + ξ_{d}^{2} + μ (a)) . \end{matrix} \end{matrix} \end{matrix}

(6.13)

The spectrum of the harmonic oscillator

\sum_{j = 1}^{n} λ_{j} (- \partial_{ξ_{j}}^{2} + ξ_{j}^{2})

\sum_{j = 1}^{n} λ_{j} (1 + 2 N)

and all its eigenvectors belong to

S (R^{n})

. Thus, the operator

π_{Δ^{a}}^{\pm, ξ}

is injective on

S (R^{n})

if, and only if,

ξ_{2 n + 1}^{2} + \dots + ξ_{d}^{2} + μ (a)

is not

\pm \sum_{j = 1}^{n} λ_{j} (1 + 2 N)

. This occurs for any

ξ \in {0}^{2 n} \times R^{d - 2 n}

if, and only if, the following condition holds

\begin{matrix} \begin{matrix} (6.15) & \begin{matrix} (6.14) & μ (a) is not in the singular set Λ_{a}, Λ_{a} = (- \infty, - \frac{1}{2} Trace | L (a) |] \cup [\frac{1}{2} Trace | L (a) |, \infty) if 2 n < d, Λ_{a} = {\pm (\frac{1}{2} Trace | L (a) | + 2 \sum_{1 \leq j \leq n} α_{j} | λ_{j} |); α_{j} \in N^{d}} if 2 n = d . \end{matrix} \end{matrix} \end{matrix}

(6.16)

In particular, the condition (ii) of Proposition 5.7 is equivalent to ( 6.14 ). Since the condition (i) is always satisfied, it follows that the Rockland condition for

Δ

is equivalent to ( 6.14 ).

Notice also that, independently of the equivalence with the Rockland condition, the condition ( 6.14 ) does not depend on the choice of the

H

-frame, because as

Λ_{a}

depends only on the eigenvalues of

L (a)

which scale in the same way as

μ (a)

under a change of

H

-frame preserving the form ( 6.2 ).

On the other hand, since the transpose

(Δ^{a})^{t} = (Δ^{t})^{a}

is given by the formula ( 6.9 ) with

μ (a)

replaced by

- μ (a)

, which has no effect on ( 6.14 ), we see that the Rockland condition for

(Δ^{t})^{a}

too is equivalent to ( 6.14 ). Therefore, we have obtained:

Proposition 6.2. The Rockland conditions for

Δ^{t}

and

Δ

a

are both equivalent to ( 6.14 ).

In particular, we see that if the principal symbol of

Δ

is invertible then the condition ( 6.14 ) holds at every point. As shown by Beals-Greiner the converse is true as well. The key result is the following.

Proposition 6.3 ([BG,Sect. 5] ). Let

U \subset R d + 1

be a Heisenberg chart near

a

and set

\begin{matrix} Ω = {(μ, x) \in C \times U; μ \notin Λ_{x}} . \end{matrix}

(6.17)

Then

Ω

is an open subset of

C \times U

and there exists

q_{μ} (x, ξ) \in C^{\infty} (Ω \times R d + 1 \ 0)

such that:

(i)

q_{μ} (x, ξ)

is analytic with respect to

μ

; (ii) We have

q_{μ} (x, λ . ξ) = λ^{- 2} q_{μ} (x, ξ)

for any

λ > 0

. (iii) For any

(μ, x) \in Ω

the symbol

q_{μ} (x, .)

inverts

| ξ^{'} |^{2} + i μ ξ_{0}

G_{x} U

, i.e., we have

\begin{matrix} q_{μ} (x, .) *^{x} (| ξ^{'} |^{2} + i μ ξ_{0}) = (| ξ^{'} |^{2} + i μ ξ_{0}) *^{x} q_{μ} (x, .) = 1 . \end{matrix}

(6.18)

More precisely,

q_{μ} (x, ξ)

is obtained from the analytic continuation of the function,

q_{μ} (x, ξ) = \int_{0}^{\infty} e^{- t μ ξ_{0}} G (x, ξ, t) d t, | ℜ μ | < \frac{1}{2} Tr | L (x) |, G (x, ξ, t) = {det}^{- \frac{1}{2}} [cosh (t | ξ_{0} | | L (x) |)] exp [- t 〈 \frac{tanh (t | ξ_{0} | | L (x) |)}{t | ξ_{0} | | L (x) |} ξ^{'}, ξ^{'} 〉] .

This implies that if the condition ( 6.14 ) is satisfied at every point

x \in U

then we get an inverse

q_{- 2} \in S_{- 2} (U \times R d + 1)

for

σ_{2} (Δ) (x, ξ) = | ξ^{'} |^{2} + i μ (x) ξ_{0}

U \times R d + 1

by letting

\begin{matrix} q_{- 2} (x, ξ) = q_{μ (x)} (x, ξ), (x, ξ) \in U \times (R d + 1 \ 0) . \end{matrix}

(6.19)

It thus follows that if ( 6.14 ) holds at every point of

M

then the principal symbol of

Δ

is invertible near any point of

M

, hence admits an inverse in

S_{- 2} (g^{*} M)

. Therefore, we get:

Proposition 6.4. A scalar sublaplacian

Δ : C^{\infty} (M) \to C^{\infty} (M)

has an invertible principal symbol if, and only if, it satisfies the condition ( 6.14 ) at every point.

Let us now extend the above results to the case of a sublaplacian

Δ : C^{\infty} (M, ℰ) \to C^{\infty} (M, ℰ)

acting on the sections of the vector bundle

ℰ

Let

a \in M

and let

X_{0}, \dots, X_{d}

be a local

H

-frame near

a

with respect to which we have

\begin{matrix} Δ = - \sum_{j = 1}^{d} X_{j}^{2} - i μ (x) X_{0} + O_{H} (1), \end{matrix}

(6.20)

where

μ (x)

is a smooth section of

End ℰ

In a suitable basis of

ℰ_{a}

the matrix of

μ (a)

is in triangular form,

\begin{matrix} μ (a) = (\begin{matrix} μ_{1} (a) & * & * \\ 0 & . . . & * \\ 0 & 0 & μ_{r} (a) \end{matrix}) . \end{matrix}

(6.21)

where

μ_{1} (a), \dots, μ_{r} (a)

denote the eigenvalues of

μ (a)

counted with multiplicity. Therefore, the model operator of

Δ

a

is of the form,

\begin{matrix} Δ^{a} = (\begin{matrix} Δ_{1}^{a} & * & * \\ 0 & . . . & * \\ 0 & 0 & Δ_{r}^{a} \end{matrix}), Δ_{j}^{a} = - [(X_{1}^{a})^{2} + \dots + (X_{1}^{a})^{2}] - i μ_{j} (a) X_{0}^{a} . \end{matrix}

(6.22)

It follows that

Δ^{a}

satisfies the Rockland condition if, and only if, so does each sublaplacian

Δ_{j}^{a}

j = 1, \dots, r

. Using Proposition 6.4 we then deduce that the Rockland condition

Δ^{a}

is equivalent to the condition,

\begin{matrix} Sp μ (a) \cap Λ_{a} = \emptyset . \end{matrix}

(6.23)

Notice that the same is true for the transpose

(Δ^{a})^{t}

. Moreover, the condition ( 6.23 ) is independent of the choice of the basis of

ℰ_{a}

or of the

H

-frame since the condition involves

μ (a)

only though its eigenvalues of

μ (a)

and the latter scale in the same way as that of

L (a)

under a change of

H

-frame preserving the form ( 6.20 ).

Next, concerning the invertibility of the principal symbol of

Δ

the following extension of Proposition 6.3 holds.

Proposition 6.5. Let

U \subset R d + 1

be a trivializing Heisenberg chart near

a

and set

\begin{matrix} Ω = {(μ, x) \in M_{r} (C) \times U; μ \notin Λ_{x}} . \end{matrix}

(6.24)

Then

Ω

is an open subset of

M_{r} (C) \times U

and there exists

q_{μ} (x, ξ) \in C^{\infty} (Ω \times R d + 1 \ 0, M_{r} (C))

so that:

(i)

q_{μ} (x, ξ)

is analytic with respect to

μ

; (ii) We have

q_{μ} (x, λ . ξ) = λ^{- 2} q_{μ} (x, ξ)

for any

λ > 0

(iii) For any

(μ, x) \in Ω

the symbol

q_{μ} (x, .)

inverts

| ξ^{'} |^{2} + i μ ξ_{0}

G_{x} U

, that is, we have

\begin{matrix} q_{μ} (x, .) *^{x} (| ξ^{'} |^{2} + i μ ξ_{0}) = (| ξ^{'} |^{2} + i μ ξ_{0}) *^{x} q_{μ} (x, .) = 1 . \end{matrix}

(6.25)

Proof. It is enough to prove that near point

(μ_{0}, x_{0}) \in Ω

there exists an open neighborhood

Ω^{'}

contained in

Ω

and a function

q_{μ} (x, ξ) \in C^{\infty} (Ω \times R d + 1 \ 0, M_{r} (C))

satisfying the properties (i), (ii) and (iii) on

Ω^{'} \times R d + 1 \ 0

To this end observe that since

Sp μ \subset \bar{D} (0, ∥ μ ∥)

for any

μ \in M_{r} (C)

, we see that if we let

K = \bar{B (0, ∥ μ_{0} ∥ + 1)}

then any

μ \in M_{r} (C)

close enough to

μ_{0}

has its spectrum contained in

K

Let

δ > 0

be small enough so that

δ < \frac{1}{2} dist (Sp μ_{0}, Λ_{x_{0}})

and define

V_{1} = Sp μ_{0} + D (0, δ)

and

V_{2} = Λ_{x_{0}} + D (0, δ)

, so that

V_{1}

and

V_{2}

are disjoint open subsets of

C

containing

Sp μ_{0}

and

Λ_{x_{0}}

respectively.

Notice that for any

μ

close enough to

μ_{0}

we have

Sp μ \subset V_{1}

. Otherwise there exists a sequence

(μ_{k})_{k \geq 1} \subset M_{r} (C)

converging to

μ_{0}

and a sequence of eigenvalues

(λ_{k})_{k \geq 1} \subset K

λ_{k} \in Sp μ_{k}

, such that

λ_{k} \notin V_{1}

for any

k \geq 1

. Since the sequence

(λ_{k})_{k \geq 1}

is bounded, we may assume that it converges to some

λ \notin V_{1}

. Necessarily

λ

is an eigenvalue of

μ_{0}

, which contradicts the fact that

λ \notin V_{1}

. Thus there exists

η_{1} > 0

so that for any

μ \in B (μ_{0}, η_{1})

we have

Sp μ \subset V_{1}

Similarly there exists

η_{2} > 0

so that for any

x \in B (x_{0}, η_{2})

we have

Sp | L (x) | \subset Sp | L (x_{0}) | + D (0, δ)

, which implies

Λ_{x} \subset Λ_{x_{0}} + D (0, δ) = V_{2}

. Therefore the open set

Ω^{'} = B (μ_{0}, η_{1}) \times B (x_{0}, η_{2})

is such that for any

(μ, x) \in Ω^{'}

we have

Sp μ \cap Λ_{x} \subset V_{1} \cap V_{2} =

, that is,

Ω^{'}

is an open neighborhood of

(μ_{0}, x_{0})

contained in

Ω

Next, let

Γ

be a smooth curve of index 1 such that the bounded connected component of

C \ Γ

contains

V_{1}

and its unbounded component contains

V_{2}

. Then we define an element of

Hol (B (μ_{0}, η_{1})) \hat{\otimes} C^{\infty} (B (x_{0}, η_{2}) \times R d + 1 \ 0)

by letting

\begin{matrix} q_{μ} (x, ξ) = \frac{1}{2 i π} \int_{Γ} q_{γ} (x, ξ) (γ - μ) d γ, (μ, x, ξ) \in Ω^{'} \times R d + 1 \ 0 . \end{matrix}

(6.26)

This function is homogeneous of degree

- 2

with respect to

ξ

and for any

(μ, x) \in Ω^{'}

we have

(6.27) q μ ( x , . ) * x ( | ξ ′ | 2 + i μ ξ 0 ) = 1 2 i π ∫ Γ q γ ( x , . ) * x ( | ξ ′ | 2 + i μ ξ 0 ) ( γ − μ ) − 1 d γ , 1 2 i π ∫ Γ [ ( γ − μ ) − 1 − i q γ ( x , . ) * ξ 0 ] d γ = 1 .

Similarly we have

(| ξ^{'} |^{2} + i μ ξ_{0}) *^{x} q_{μ} (x, .) = 1

. Thus

q_{μ} (x, ξ)

satisfies the properties (i), (ii) and (iii) on

Ω^{'} \times R d + 1 \ 0

. The proof is thus complete. □

In the same way as Proposition 6.3 in the scalar case, Proposition 6.5 implies that when the condition ( 6.23 ) holds everywhere the principal symbol of

Δ

admits an inverse in

S_{- 2} (g^{*} M, ℰ)

. We have thus proved:

Proposition 6.6. 1) At every point

a \in M

the Rockland conditions for

Δ

and

Δ^{t}

are equivalent to ( 6.23 ).

2) The principal symbol of

Δ

is invertible if, and only if, the condition ( 6.23 ) holds everywhere. Moreover, when the latter occurs

Δ

admits a parametrix in

Ψ^{- 2} H (M, ℰ)

and is hypoelliptic with loss of 1 derivative.

7 Examples of hypoelliptic operators on Heisenberg manifolds

In this section we explain how the previous results of this paper can used to deal with the hypoellipticity for the main geometric operators on Heisenberg manifolds: Hörmander's sum of squares, Kohn Laplacian, horizontal sublaplacian and contact Laplacian. In particular, the treatment in [BG] of the Kohn Laplacian and we establish a criterion for the invertibility of the horizontal sublaplacian, which has not been done before.

7.1 Hörmander's sum of squares.

Let

X_{1}, \dots, X_{m}

be (real) vector fields on a manifold

M^{d + 1}

and consider the sum of squares,

\begin{matrix} Δ = - (X_{1}^{2} + \dots + X_{m}^{2}) . \end{matrix}

(7.1)

By a celebrated theorem of Hörmander [H¨2] the operator

Δ

is hypoelliptic provided that thefollowing bracket condition is satisfied: the vector fields

X_{0}, X_{1}, \dots, X_{m}

together with their successiveLie brackets

[X_{j_{1}}, [X_{j_{2}}, \dots, X_{j_{1}}] \dots]]

span the tangent bundle

T M

at every point. When

X_{1}, \dots, X_{m}

span a hyperplane bundle

H

the operator

Δ

is a sublaplacian with realcoefficients and the bracket condition reduces to

H + [H, H] = T M

, which is equivalent to thenon-vanishing of the Levi form of

(M, H)

. More generally, given a vector bundle

ℰ

, the theorem of Hörmander holds for sublaplacians

Δ : C^{\infty} (M, ℰ) \to C^{\infty} (M, ℰ)

of the form

\begin{matrix} Δ = - (\nabla_{X_{1}}^{2} + \dots + \nabla_{X_{m}}^{2}) + O_{H} (1), \end{matrix}

(7.2)

where

\nabla

is a connection on

ℰ

. In particular, if

M

is endowed with a positive density and

ℰ

with a Hermitian metric, this includes the selfadjoint sum of squares,

\begin{matrix} Δ = \nabla_{X_{1}}^{*} \nabla_{X_{1}} + \dots + \nabla_{X_{m}}^{*} \nabla_{X_{m}} . \end{matrix}

(7.3)

In fact, if

Δ

is a sublaplacian of the form ( 7.2 ) then in ( 6.1 ) the matrix

μ (x)

vanishes, so that ( 6.23 ) holds if, and only if, the Levi form does not vanish at

x

. Therefore, we obtain:

Proposition 7.1. Let

Δ : C^{\infty} (M, ℰ) \to C^{\infty} (M, ℰ)

be a (generalized) sum of squares of the form ( 7.2 ). Then: 1) At a point

x \in M

the operators

Δ

and

Δ^{t}

satisfies the Rockland condition if, and only if, the Levi form

ℒ

does not vanish at

x

2) The principal symbol of

Δ

is invertible if, and only if, the Levi form is non-vanishing.

In particular, when the latter occurs

Δ

admits a parametrix in

Ψ^{- 2} H (M, ℰ)

and is hypoelliptic with loss of one derivative.

In particular, since the nonvanishing of the Levi form is equivalent to the bracket condition

H + [H, H] = T M

, we see that, the special case of Heisenberg manifolds, we recover the hypoellipticity result of [H¨2] for sums of squares.

7.2 The Kohn Laplacian

In [KR] Kohn-Rossi showed that the Dolbeault complex on a bounded complex domain induces on its boundary a horizontal complex of differential forms. This was later extended by Kohn [Koh1] to the general setting of a CR manifold

M^{2 n + 1}

as follows.

Let

M^{2 n + 1}

be a CR manifold with CR bundle

T_{1, 0} \subset T_{C} M

and set

T_{0, 1} = \bar{T_{1, 0}}

. Then the subbundle

H = ℜ (T_{1, 0} \oplus T_{0, 1}) \subset T M

admits an integrable complex structure and the splitting

H \otimes C = T_{1, 0} \oplus T_{0, 1}

gives rise to a decomposition

Λ H^{*} \otimes C = \oplus_{0 \leq p, q \leq n} Λ^{p, q}

, where

(p, q)

is called a bidegree of a form with values in

Λ^{p, q}

Assume that

T_{C} M

is endowed with a Hermitian metric such that

T_{1, 0}

and

T_{0, 1}

are orthogonal subspaces and complex conjugation is an (antilinear) isometry. This Hermitian metric gives rise to a Hermitian metric on

Λ^{*} T_{C}^{*} M

with respect to which the decomposition

Λ H^{*} \otimes C = \oplus_{0 \leq p, q \leq n} Λ^{p, q}

becomes orthogonal. Let

Π_{p, q} : Λ^{*} T_{C}^{*} M \to Λ^{p, q}

be the orthogonal projection onto

Λ^{p, q}

. Then the Kohn-Rossi operator

{\bar{\partial}}_{b} : C^{\infty} (M, Λ^{p, q}) \to C^{\infty} (M, Λ^{p, q + 1})

is given by

\begin{matrix} {\bar{\partial}}_{b} η = Π_{p, q + 1} (d η), η \in C^{\infty} (M, Λ^{p, q}) . \end{matrix}

(7.4)

Since the integrability of

T_{1, 0}

implies that

{\bar{\partial}}_{b}^{2} = 0

, this yields chain complexes

{\bar{\partial}}_{b} : C^{\infty} (M, Λ^{p, *}) \to C^{\infty} (M, Λ^{p, * + 1})

Endowing

M

with a smooth density

ρ > 0

we let

{\bar{\partial}}_{b}^{*}

denote the formal adjoint of

{\bar{\partial}}_{b}

. Then the Kohn Laplacian is

\begin{matrix} □_{b} = ({\bar{\partial}}_{b} + {\bar{\partial}}_{b}^{*})^{2} = {\bar{\partial}}_{b}^{*} {\bar{\partial}}_{b} + {\bar{\partial}}_{b} {\bar{\partial}}_{b}^{*} . \end{matrix}

(7.5)

The Kohn Laplacian is a sublaplacian (see [FS,Sect. 13] , [BG,Sect. 20] ) and it was shown by Kohn [Koh1] that under the condition

Y (q)

this operator is hypoelliptic with loss of one derivative when acting on

(p, q)

-forms.

The condition

Y (q)

means that if for

x \in M

we let

(r (x) - κ (x), κ (x), n - r (x))

be the signature of

L_{θ}

x

, so that

r (x)

is the rank of

L_{θ}

and

κ (x)

the number of its negative eigenvalues, then the condition

Y (q)

is satisfied at

x \in M

when we have

\begin{matrix} q \notin {κ (x), κ (x) + 1, \dots, κ (x) + n - r (x)} \cup {r (x) - κ (x), r (x) - κ (x) + 1, \dots, n - κ (x)} . \end{matrix}

(7.6)

For instance, when

M

κ

-strictly pseudoconvex, the

Y (q)

-condition exactly means that we must have

q \neq κ

and

q \neq n - κ

In fact, as shown in [BG,Sect. 21] , at every point

a \in M

the condition

Y (q)

is equivalent to the condition ( 6.23 ) for the Kohn Laplacian acting on

(p, q)

-forms. Therefore, using Proposition 6.6 we immediately get:

Proposition 7.2. Let

□_{b} : C^{\infty} (M, Λ^{p, q}) \to C^{\infty} (M, Λ^{p, q})

be the Kohn Laplacian acting on

(p, q)

-forms.

1) At a point

x \in M

the Rockland condition for

□_{b}

is equivalent to the condition

Y (q)

2) The principal symbol of

□_{b}

is invertible if, and only if, the condition

Y (q)

is satisfied at every point. In particular, when the latter occurs

□_{b}

admits a parametrix in

Ψ^{- 2} H (M, Λ^{p, q})

and is hypoelliptic with loss of one derivative.

Remark 7.3. The proof of the second part of the statement in [BG,Sect. 21] is not quite complete, because Beals-Greiner claim that diagonalizing the leading part of the Kohn Laplacian allows us to use to apply the results for scalar sublaplacians. While this property is true in case of a Levi Metric (see [FS] ), it may fail for a general metric on

T_{C} M

since a smooth eigenframe needs not exists For instance, for the Kohn Laplacian the eigenvalues of the matrix

μ (x)

in ( 6.1 ) with respect to an orthonormal

H

-frame of

T M

are given in terms of eigenvalues of the Levi form (see Eq. (21.31) in [BG] ), but the latter need not depend smoothly on

x

(unless the metric on

T_{C} M

is a Levi metric).

Therefore, in order to deal with the Kohn Laplacian acting on forms, we really need to use the version for sublaplacians acting on section of vector bundles, provided by Proposition 6.6 , but not deal with in [BG] .

Remark 7.4. The

Y (q)

-condition is only a sufficient condition for the hypoellipticity of

□_{b}

, as the latter may be hypoelliptic when the

Y (q)

-condition fails (e.g. [Koh2] , [Ko] , [Ni] ).

7.3 The horizontal sublaplacian on a Heisenberg manifold

Let

(M^{d + 1}, H)

be a Heisenberg manifold endowed with a Riemannian metric and let

Λ_{C}^{*} H^{*} = \oplus_{k = 0}^{d} Λ_{C}^{k} H^{*}

be the (complexified) bundle of horizontal forms. Then the horizontal sublaplacian

Δ_{b} : C^{\infty} (M, Λ_{C}^{*} H^{*}) \to C^{\infty} (M, Λ_{C}^{*} H^{*})

is given by

\begin{matrix} Δ_{b} = d_{b}^{*} d_{b} + d_{b} d_{b}^{*}, d_{b} α = π_{b} (d α), \end{matrix}

(7.7)

where

π_{b}

denotes the orthogonal projection of

Λ_{C}^{*} T^{*} M

onto

Λ_{C}^{*} H^{*}

This operator was first introduced by Tanaka [Ta] for strictly pseudoconvex CR manifolds, but versions of this operator acting on functions were independently defined by Greenleaf [Gr] and Lee [Le] .

Moreover, it can be shown that

d_{b}^{2} = 0

if, and only if, the subbundle

H

is integrable, so in general

Δ_{b}

is not the Laplacian of a chain complex.

As we shall see to determine under which condition the principal symbol of

Δ_{b}

is invertible the relevant condition to look at is the

X (k)

-condition below.

Definition 7.5. For

x \in M

let

2 r (x)

denote the rank of the Levi form

ℒ

x

. Then we say that

ℒ

satisfies the condition

X (k)

x

when we have

For instance, the condition

X (0)

is satisfied if, and only if, the Levi form does not vanish. Also, if

M^{2 n + 1}

is a contact manifold or a nondegenerate CR manifold then the Levi form is everywhere nondegenerate, so

r (x) = 2 n

and the

X (k)

-condition becomes

k \neq n

. In any case, we have:

Proposition 7.6. Let

Δ_{b} : C^{\infty} (M, Λ_{C}^{k} H^{*}) \to C^{\infty} (M, Λ_{C}^{k} H^{*})

be the horizontal sublaplacian acting on horizontal forms of degree

k

1) At a point

x \in M

Δ_{b}

X (k)

Raphaël Ponge