Convergence Criteria for a Hopfield-type Neural Network

Raveen GoundarCorresponding Author. Email: raveen g@hotmail.com and Jito Vanualailai Department of Mathematics and Computing Science, University of the South Pacific, Suva, Fiji. 2000 Mathematics Sub ject Classification: 34D20, 92B20.

Abstract
Motivated by recent applications of the Lyapunov's method in artificial neural networks, which could be considered as dynamical systems for which the convergence of the system trajectories to equilibrium states is a necessity. We re-look at a well-known Krasovskii's stability criterion pertaining to a non linear autonomous system. Instead, we consider the components of the same autonomous system with the help of the elements of Jacobian matrix J(x), thus proposing much simpler convergence criteria via the method of Lyapunov. We then apply our results to artificial neural networks and discuss our results with respect to recent ones in the field.
Keywords and Phrases: Lyapunov Stability, Hopfield-Tank Neural Networks

1 Introduction

The Direct Method of Lyapunov, which utilizes energy-like functions called Lyapunov functions, is now a well-entrenched technique in the qualitative analysis of mathematical systems governed by differential equations. A flurry of activities by mathematicians, particularly within the period of early 1940s and the late 1960s, extended the work of Lyapunov to produce results that are now indispensable in many applications. (A good modern review of the Lyapunov method and its many applications is by Sastry [1] .) This paper is motivated to a large extent by modern applications of the Lyapunov method, especially in the field of artificial neural networks.
We start by considering the autonomous system of the form
x ′ ( t ) = g ( x ) , x ( t 0 ) = x 0 . (1)
Throughout the paper, guided by a well-known result of Krasovskii, we will strive to portray a simple and flexible method of proposing a stability criterion for system ( 1 ). We conclude by considering an application in artificial neural networks.
Throughout the article, we suppose that, in system ( 1 ), g ∈ C [ R n , R n ]   , and is smooth enough to guarantee existence, uniqueness and continuous dependence of solutions x ( t ) = x ( t ; x 0 )   , with x = ( x 1 , … , x n ) T   . The following definition and theorems of Lyapunov will be used in this article.
(We will use those in Glendenning [2] ).
Definition 1. Suppose that the origin, x = 0   , is an equilibrium point for system ( 1 ). Let D   be an open neighborhood of 0   and V : D → R   be a continuously differentiable function. Then we can define the derivative of V   along trajectories by differentiating V   with respect to time using the chain rule, so V ′ ( x ) = d V ( x ) d t = x ′ ⋅ ∇ V ( x ) = g ( x ) ⋅ ∇ V ( x ) = ∑ i = 1 n g i ( x ) ∂ V ( x ) ∂ x i ,   where the subscripts denote the components of g   and x   . Then V   is a Lyapunov function on D   iff
  • (i) V   is continuously differentiable on D   ;
  • (ii) V ( 0 ) = 0   and V ( x ) > 0   for all x ∈ D \ { 0 }   ;
  • (iii) V ′ ( x ) ≤ 0   for all x ∈ D   .
Theorem 1 (Lyapunov's Stability Theorem). Let x = 0   be an equilibrium point for system ( 1 ) and D ⊂ R n   be a domain containing x = 0   .
Let V ( x )   be a Lyapunov function on an open neighborhood of D   , then x = 0   is stable.
Theorem 2 (Lyapunov's Asymptotic Stability Theorem). Let x = 0   be an equilibrium point for system ( 1 ) and D ⊂ R n   be a domain containing x = 0   . Let V ( x )   be a Lyapunov function on an open neighborhood of D   . If V ′ ( 0 ) = 0   and V ′ ( x ) < 0   for all x ∈ D \ { 0 }   , then x = 0   is asymptotically stable.
Theorem 3 (Lyapunov's Theorem of Global Asymptotic Stability). Let x = 0   be a equilibrium point for system ( 1 ) and let V ( x )   be a Lyapunov function for all x ∈ R n   . If x = 0   is asymptotically stable and V ( x )   is radially unbounded, then x = 0   is globally asymptotically stable.
We carry the assumption that g ( 0 ) ≡ 0   so that 0   is the zero solution of ( 1 ).

2 Convergence Criteria

In 1954, Krasovskii [3] established an asymptotic stability criterion that avoided the linearization principle, and in the process established a method of estimating the extent of asymptotic stability region for a nonlinear systems.
He assumed that g ∈ C ′ [ R n , R n ]   and g ( 0 ) = 0   . Then system ( 1 ) can be written as x ′ ( t ) = ∫ 0 1 J ( s x ) x d s   where J   is the Jacobian matrix J ( x ) = ∂ g ( x ) ∂ x .   The following result by Krasovskii is a fundamental one in control theory.
Theorem 4 (Krasovskii [3] ). Let g ∈ C ′ [ R n , R n ]   and g ( 0 ) = 0   . If there exists a constant positive definite symmetric matrix P   such that x T [ P J ( x ) + J T ( x ) P ] x   is a negative definite function, then the zero solution of system ( 1 ) is globally asymptotically stable.
For our purpose, we need a criterion that explicitly uses each component of system ( 1 ). Thus, using the elements of Jacobian matrix; J i j ( x )   , we define
D ( x ) = [ d i j ( x ) ] n × n (2)
where d i j ( x ) = ∫ 0 1 J i j ( s x ) d s = ∫ 0 1 ∂ g i ( s x ) ∂ ( s x j ) d s ,   such that system ( 1 ) can be rewritten as
x ′ ( t ) = D ( x ) x . (3)
A decoupled form for the i   -th component of system ( 3 ) is
x i ′ ( t ) = d i i ( x ) x i + ∑ j = 1 j ≠ i n d i j ( x ) x j . (4)
Remark 1. Note that in ( 4 ), the term d i j ( x ) x j   , for i , j = 1 , … , n   , is continuously differentiable with respect to x ∈ R n   for the simple reason that D ( x ) x = g ( x )   and g ∈ C ′ [ R n , R n ]   .
The following result of ours, guarantees the convergence criteria for autonomous system ( 1 ).
Theorem 5. Let g ∈ C ′ [ R n , R n ]   and g ( 0 ) = 0   . Let β i ( x ) = d i i ( x ) + 1 2 ∑ j = 1 j ≠ i n ( | d i j ( x ) | + | d j i ( x ) | ) .   Define D = { x ∈ R n : ∥ x ∥ ≤ M }   for some M > 0   and assume that d i j ( x ) x i   are continuous on R n   for i , j = 1 , … , n   , such that i ≠ j   .
Then the zero solution of ( 1 ) is
  • (a) stable if − ∞ < β i ( x ) ≤ 0   for i = 1 , 2 , … , n   and x ∈ D   .
  • (b) asymptotically stable if − ∞ < β i ( x ) < 0   for i = 1 , 2 , … , n   and x ∈ D   .
  • (c) globally asymptotically stable if − ∞ < β i ( x ) < 0   for all x ∈ R n   .
  • Proof. Consider V ( x ) = 1 2 ∑ i = 1 n x i 2   as a tentative Lyapunov function for system ( 1 ). We have, along a solution of ( 1 ),  1 
    d d t [ V ] ( ) = 1 2 ∑ i = 1 n d d t [ x i 2 ] = ∑ i = 1 n x i x i ′ ( t )
    = ∑ i = 1 n x i [ d i i ( x ) x i + ∑ j = 1 j ≠ i n d i j ( x ) x j ]
    = ∑ i = 1 n [ d i i ( x ) x i 2 + ∑ j = 1 j ≠ i n d i j ( x ) x j x i ]
    = ∑ i = 1 n [ d i i ( x ) x i 2 + 1 2 ∑ j = 1 j ≠ i n [ d i j ( x ) + d j i ( x ) ] x j x i ]
    ≤ ∑ i = 1 n [ d i i ( x ) x i 2 + 1 4 ∑ j = 1 j ≠ i n [ | d i j ( x ) | + | d j i ( x ) | ] ( x j 2 + x i 2 ) ]
    = ∑ i = 1 n [ d i i ( x ) + 1 2 ∑ j = 1 j ≠ i n [ | d i j ( x ) | + | d j i ( x ) | ] ] x i 2 (5)
    = ∑ i = 1 n β i ( x ) x i 2 . (6)
    Expanded form of system ( 5 ) is d V d t ≤ ∑ i = 1 n [ d i i ( x ) x i x i + 1 2 ∑ j = 1 j ≠ i n [ | d i j ( x ) x i x i | + | d j i ( x ) x i x i | ] ] .   By Remark  1 , the first and third terms of system ( 5 ) are continuous on R n   , and by assumption of Theorem  5 , the second term is also continuous on R n   .
    Hence V ( x )   is continuous on R n   . Since
    V ( x ) = 1 2 ∑ i = 1 n x i 2 , (7)
    we have therefore, V ( 0 ) = 0   and V ( x ) > 0   for all x ∈ R n \ { 0 }   . From equation ( 6 ),
    V ′ ( x ) ≤ ∑ i = 1 n β i ( x ) x i 2 (8)
    and by condition (a) of Theorem  5 , we have V ′ ( x ) ≤ 0   for all x ∈ D   .
    Hence by Theorem  1 , the zero solution of system ( 1 ) is stable. Moreover, by condition (b) of Theorem  5 , equation ( 8 ) implies V ′ ( 0 ) = 0   and V ′ ( x ) < 0   for all x ∈ D \ { 0 }   . Hence by Theorem  2 , the zero solution of system ( 1 ) is asymptotically stable. Furthermore, by condition (c) of Theorem  5 , equation ( 8 ) implies V ′ ( 0 ) = 0   and V ′ ( x ) < 0   for all x ∈ R n   . Note that ( 7 ) implies V ( x ) → ∞   as ∥ x ∥ → ∞   , thus V ( x )   is radially unbounded. Hence by Theorem  3 , the zero solution of system ( 1 ) is globally asymptotically stable.
Let us consider some examples to show the applicability of Theorem  5 .
Example 1. We consider the following two-dimensional system
[ x 1 ′ ( t ) x 2 ′ ( t ) ] = [ − 2 x 1 + x 2 2 x 1 2 − 2 x 2 ] , (9)
with x 1 ( t 0 ) = x 10   and x 2 ( t 0 ) = x 20   . In the form of system ( 3 ), system ( 9 ) can be written as
[ x 1 ′ ( t ) x 2 ′ ( t ) ] = [ − 2 x 2 x 1 − 2 ] [ x 1 x 2 ] .
The assumption of Theorem  5 is satisfied since d 12 ( x ) x 1 = d 21 ( x ) x 2 = x 1 x 2 .   Next we shall check condition (a) of Theorem  5 . We have
β 1 ( x ) = d 11 ( x ) + 1 2 ( | d 12 ( x ) | + | d 21 ( x ) | )
= − 2 + 1 2 ( | x 2 | + | x 1 | ) .
Solving the inequality β 1 ( x ) < 0   , we have | x 1 | + | x 2 | < 4 ,   and `squaring' both sides gives x 1 2 + x 2 2 + 2 | x 1 | | x 2 | < 16 .   Now x 1 2 + x 2 2 + 2 | x 1 | | x 2 | < x 1 2 + x 2 2 + 2 × 1 2 ( x 1 2 + x 2 2 ) = 2 x 1 2 + 2 x 2 2 .   Then let 2 x 1 2 + 2 x 2 2 < 16   from which x 1 2 + x 2 2 < 8 .   Similarly solving β 2 ( x ) < 0   , we have β 2 ( x ) = d 22 ( x ) + 1 2 ( | d 21 ( x ) | + | d 12 ( x ) | ) < 0 ,   which gives
− 2 + 1 2 ( | x 1 | + | x 2 | ) < 0 . (10)
Further simplification of ( 10 ) gives us x 1 2 + x 2 2 < 8 .   Therefore, let D = { x ∈ R 2 : ∥ x ∥ < 8 } .   Hence by condition (a) of Theorem  5 , the zero solution of system ( 9 ) is asymptotically stable.
Example 2. We consider the following two-dimensional system
[ x 1 ′ ( t ) x 2 ′ ( t ) ] = [ − 4 x 1 + x 1 sech ( x 1 ) + 4 x 2 − x 1 − 6 x 2 − x 2 cos ( x 2 ) ] , (11)
which can be written in the form of system ( 3 ) as
[ x 1 ′ ( t ) x 2 ′ ( t ) ] = [ − 4 + sech ( x 1 ) 4 − 1 − 6 − cos ( x 2 ) ] [ x 1 x 2 ] .
The assumption of Theorem  5 is satisfied since d 12 ( x ) x 1 = 4 x 1   and d 21 ( x ) x 2 = − x 2   . Next we shall check condition (c) of Theorem  5 .
We have
β 1 ( x ) = d 11 ( x ) + 1 2 ( | d 12 ( x ) | + | d 21 ( x ) | )
= − 4 + sech ( x 1 ) + 1 2 ( | 4 | + | − 1 | )
= sech ( x 1 ) − 3 2 ≤ 1 − 3 2 = − 1 2 < 0 .
Similarly, we have
β 2 ( x ) = d 22 ( x ) + 1 2 ( | d 21 ( x ) | + | d 12 ( x ) | )
= − 6 − cos ( x 2 ) + 1 2 ( | − 1 | + | 4 | )
= − 7 2 − cos ( x 2 ) ≤ − 7 2 + 1 = − 5 2 < 0 .
Since both β 1 ( x ) < 0   and β 2 ( x ) < 0   for all x ∈ R 2   hence by condition (c) of Theorem  5 , the zero solution of system ( 11 ) is globally asymptotically stable.

3 Application in Artificial Neural Networks

Artificial neural networks (ANNs) can be considered as dynamical systems with several equilibrium states. An essential operating condition for a neural network is that all system trajectories must converge to the equilibrium states. (A good overview of the concepts associated with biological neural networks is given in [5] ).
We will consider an ANN that is described thoroughly in Lakshmikantham et al. [6] , and provide a stability criteria using Theorem  5 . The ANN in question has n   units. To the i   th unit, we associate its activation state at time t   , a real number x i = x i ( t )   ; an output function μ i   ; a fixed bias θ i   ; and an output signal R i = μ i ( x i + θ i )   . The weight or connection strength on the line from unit j   to unit i   is a fixed real number W i j   . When W i j = 0   , there is no transmission from unit j   to unit i   . The incoming signal from unit j   to unit i   is S i j = W i j R j   . In addition, there can be a vector I   of any number of external inputs feeding into some or all units, so that we may write I = ( I 1 , … , I m ) T   .
An ANN with fixed weights is a dynamical system: given initial values of the activation of all units, the future activations can be computed. The future activation states are assumed to be determined by a system of n   differential equations, the i   th equation of which is
x i ′ ( t ) = G i ( x i , S i 1 , … , S i n , I ) = G i ( x i , W i 1 R 1 , … , W i n R n , I )
= G i ( x i ; W i 1 μ 1 ( x 1 + θ 1 ) , … , W i n μ n ( x n + θ n ) ; I 1 , … , I m ) . (12)
With W i j   , θ i   and I k   assumed known, we can write ( 12 ) as
x i ′ ( t ) = g i ( x 1 , … , x n ) , (13)
or in vector notation
x ′ ( t ) = g ( x ) , (14)
where g   is a vector on Euclidean space R n   whose i   th element is g i   given in ( 13 ). We assume that g   is continuously differentiable and satisfies the usual theorems on existence, continuity and uniqueness of solutions. Thus, since g ∈ C ′ [ R n , R n ]   , we can define D ( x )   as in ( 2 ) but using g   in ( 14 ). Hence, if g ( 0 ) ≡ 0   , then system ( 14 ) can be written as x ′ ( t ) = D ( x ) x , x ( t 0 ) = x 0 ,   the i   th component of which in a decoupled form is x i ′ ( t ) = d i i ( x ) x i + ∑ j = 1 j ≠ i n d i j ( x ) x j .   First, we state a comparable result by Lakshmikantham et al. [6] , page 152, who used the concept of vector Lyapunov functions.
Theorem 6 (Lakshmikantham, Matrosov and Sivasundaram [6] ). Let g ∈ C ′ [ R n , R n ]   and g ( 0 ) = 0   . Let
β i ( x ) = d i i ( x ) + ∑ j = 1 j ≠ i n | d i j ( x ) | . (15)
Suppose that
β i ( x ) < 0 if x i 2 ≥ x j 2 , (16)
for i , j = 1 , … , n   and x ∈ R n   , x ≠ 0   . Then the zero solution of ( 14 ) is globally asymptotically stable.
If we apply condition (b) of Theorem  5 , then we obtain a simpler convergence criteria.
Theorem 7. Let g ∈ C ′ [ R n , R n ]   and g ( 0 ) = 0   . Let β i ( x ) = d i i ( x ) + 1 2 ∑ j = 1 j ≠ i n ( | d i j ( x ) | + | d j i ( x ) | ) .   Define D = { x ∈ R n : ∥ x ∥ ≤ M }   for some M > 0   and assume that d i j ( x ) x i   are continuous on R n   for i , j = 1 , … , n   , such that i ≠ j   . Then the zero solution of ( 14 ) is asymptotically stable if − ∞ < β i ( x ) < 0   for i = 1 , 2 , … , n   and x ∈ D   .
Thus, the application of Theorem  5 to artificial neural network, considering system ( 14 ), gives us a simpler criterion guaranteeing asymptotic stability as showed by Theorem  6 . Hence the strong condition x i 2 ≥ x j 2   that appears in Theorem  6 is not necessary.
Next, we look at a specific case of ( 14 ). The specific ANN is of the additive type and is often referred to as the Hopfield-Tank ANN, a much studied class of network dynamics [7] . It is described by the nonlinear differential equation
x i ′ ( t ) = − a i x i ( t ) + ∑ j = 1 n W i j μ j ( x j ( t ) + θ j ) + I i ( t )
= − a i x i ( t ) + ∑ j = 1 n W i j ν j ( x j ( t ) ) + I i ( t ) , (17)
where a i > 0   is the constant decay rate , I i ( t )   is the external input (to the i   th neuron) defined almost everywhere on [ 0 , ∞ )   and ν i   is the suppressed notation for the fixed θ i   by having θ i   incorporated into μ i   . The function ν i   is called the neuron activation function .
Now, define A = diag ( − a 1 , … , − a n )   , x = ( x 1 , … , x n ) T   , h i ( x ) = ∑ j = 1 n W i j ν j ( x j ) with h ( x ) = ( h 1 ( x ) , … , h n ( x ) ) T ,   and u ( t ) = ( I i ( t ) , … , I n ( t ) ) T   . Then ( 17 ) is the i   th component of the system
x ′ ( t ) = A x + h ( x ) + u ( t ) , x ( t 0 ) = x 0 . (18)
When the external input vector, u   , is zero, the nonautonomous system ( 18 ) reduces to the autonomous system
x ′ ( t ) = A x + h ( x ) , x ( t 0 ) = x 0 . (19)
For this, we assume that x * = ( x 1 * , … , x n * ) T   is an equilibrium point, so that A x * + h ( x * ) = 0   . By translating the origin, 0   , to this equilibrium point, we can make 0   an equilibrium point. In this case, h ( 0 ) ≡ 0   . Since this is of great notational help, we will henceforth consider 0   as an equilibrium point or zero solution of ( 19 ).
Let us next assumed that h ∈ C ′ [ R n , R n ]   . Then using the elements of Jacobian matrix, J i j ( x )   , we define F ( x ) = [ f i j ( x ) ] n × n where f i j ( x ) = ∫ 0 1 J i j ( x ) d s = ∫ 0 1 ∂ h i ( s x ) ∂ ( s x j ) d s ,   hence system ( 19 ) can be rewritten as
x ′ ( t ) = A x + F ( x ) x = [ A + F ( x ) ] x . (20)
The i   th component of ( 20 ) in a decoupled form is x i ′ ( t ) = [ − a i + f i i ( x ) ] x i ( t ) + ∑ j = 1 j ≠ i n f i j ( x ) x j .   Thus the following theorem is an application of our result; Theorem  5 .
Theorem 8. Let h ∈ C ′ [ R n , R n ]   and h ( 0 ) = 0   . Let β i ( x ) = − a i i + f i i ( x ) + 1 2 ∑ j = 1 j ≠ i n ( | f i j ( x ) | + | f j i ( x ) | ) .   Define D = { x ∈ R n : ∥ x ∥ ≤ M }   for some M > 0   and assume that f i j ( x ) x i   are continuous on R n   for i , j = 1 , … , n   , such that i ≠ j   .
Then the zero solution of ( 19 ) is
  • (a) stable if − ∞ < β i ( x ) ≤ 0   for i = 1 , 2 , … , n   and x ∈ D   .
  • (b) asymptotically stable if − ∞ < β i ( x ) < 0   for i = 1 , 2 , … , n   and x ∈ D   .
  • (c) globally asymptotically stable if − ∞ < β i ( x ) < 0   for all x ∈ R n   .
  • Proof. Applying Theorem  5 to system ( 19 ), and hence to system ( 20 ), with D ( x ) = A + F ( x )   , d i i ( x ) = − a i + f i i ( x )   and d i j ( x ) = f i j ( x )   , we easily obtain the conclusion of Theorem  8 .
Let us consider one example of Theorem  8 .
Example 3. Let us consider two-neural autonomous system.
[ x 1 ′ ( t ) x 2 ′ ( t ) ] = [ − a 1 0 0 − a 2 ] [ x 1 x 2 ] + [ h 1 ( x ) h 2 ( x ) ] (21)
with x 1 ( t 0 ) = x 10 , x 2 ( t 0 ) = x 20 , 0 ≤ t 0 ≤ t   , where,
a 1 = 10 , a 2 = 10 ,
h 1 ( x ) = B 11 ν 1 ( x 1 ) + B 12 ν 2 ( x 2 ) = − 3 x 1 + x 2 − tanh ( 3 x 1 ) ,
h 2 ( x ) = B 21 ν 1 ( x 1 ) + B 22 ν 2 ( x 2 ) = x 1 − x 2 + 1 5 tanh ( 3 x 2 ) .
In the form of system ( 20 ), system ( 21 ) can be written as
[ x 1 ′ ( t ) x 2 ′ ( t ) ] = ( [ − 10 0 0 − 10 ] + [ − 3 − τ ( x 1 ( t ) ) 1 1 − 1 + 1 5 τ ( x 2 ( t ) ) ] ) [ x 1 x 2 ] ,
where for i = 1 , 2 ,   we define τ ( x i ( t ) ) = { tanh ( 3 x i ) x i x i ≠ 0 , 3 x i = 0 ,   noting that 0 < τ ( x i ) ≤ 3   for all x i ∈ R 2   . The assumption of Theorem  8 is satisfied since f 12 ( x ) x 1 = x 1   and f 21 ( x ) x 2 = x 2   . Now we shall check condition (c) of Theorem  8 . We have
β 1 ( x ) = − a 1 + f 11 ( x ) + 1 2 ( | f 12 ( x ) | + | f 21 ( x ) | )
= − 10 − 3 − τ ( x 1 ( t ) ) + 1 2 ( | 1 | + | 1 | )
= − 12 − τ ( x 1 ( t ) ) (22)
< − 12
for all x ∈ R 2 \ { 0 }   and
β 2 ( x ) = − a 2 + f 22 ( x ) + 1 2 ( | f 21 ( x ) | + | f 12 ( x ) | )
= − 10 − 1 + 1 5 τ ( x 2 ( t ) ) + 1 2 ( | 1 | + | 1 | )
= − 10 + 1 5 τ ( x 2 ( t ) ) (23)
< − 10 + 3 5 = − 47 5
for all x ∈ R 2 \ { 0 }   . Clearly, both β 1 ( x ) < 0   and β 2 ( x ) < 0   for all x ∈ R 2 \ { 0 }   .
Next, we shall check the condition on β i ( x )   for x = 0   , where i = 1 , 2   .
From ( 22 ), we have β 1 ( x ) = − 12 − τ ( x 1 ( t ) ) .   Therefore, β 1 ( 0 ) = − 12 − 3 = − 15 .   Similarly, from ( 23 ), we have β 2 ( x ) = − 10 + 1 5 τ ( x 2 ( t ) ) .   Therefore, β 2 ( 0 ) = − 10 + 3 5 = − 47 5 .   Since β 1 ( x ) < 0   and β 2 ( x ) < 0   for all x ∈ R 2   , therefore, by condition (c) of Theorem  8 , the zero solution of system ( 21 ) is globally asymptotically stable.

4 Conclusion

We have established the criteria for stability, asymptotic stability and global asymptotic stability for a non linear autonomous system via the method of Lyapunov. We have also considered the usefulness of our main results by application of it to artificial neural networks.
Further research in this direction is being carried out, considering a non autonomous system, wherein the external input source is not assumed to be zero. Determining the convergence criteria for a non autonomous system and to measure its rate of convergence will be of grandness in applications to artificial neural networks.
References

  1. S. Sastry, Nonlinear Systems: Analysis, Stability and Control, New York: Springer-Verlag, 1999.
  2. P. Glendenning, Stability, instability and chaos: an introduction to the theory of nonlinear differential equations, Cambridge University Press, INC., New York.
  3. N. N. Krasovskii, “On the stability in the large of a system of nonlinear differential equations”, Prikl. Mat. Meh., vol 18, pp. 735–737, 1954.
  4. J. Vanualailai, T. Soma and S. Nakagiri, “Convergence of Solutions and Practical Stability of Hopfield-type Neural Networks with Time-Varying External Inputs”, Nonlinear Studies, vol 9, pp. 109–122, 2002.
  5. M. A. Arbib (editor), The Handbook of Brain Theory and Neural Networks, London: MIT Press, 1995.
  6. V. Lakshmikantham, V. M. Matrosov and S. Sivasundaram, Vector Lyapunov Functions and the Stability Analysis of Nonlinear Systems, The Netherlands: Kluwer Academic Publishers, 1991.
  7. J. J. Hopfield and D. W. Tank, “Computing with neural circuits: A Model,” Science, vol. 233, pp. 625–632, 1986.