CME 345: MODEL REDUCTION - Nonlinear Systems

CME 345: MODEL REDUCTION

Methods for Nonlinear Systems

David Amsallem & Charbel Farhat Stanford University cfarhat@stanford.edu

David Amsallem & Charbel Farhat Stanford University cfarhat@stanford.edu

CME 345: MODEL REDUCTION - Nonlinear Systems

CME 345: MODEL REDUCTION - Nonlinear Systems

Outline

1 Nonlinear Dynamical Systems 2 Linear Approximation of the Nonlinear Function 3 Piece-Wise Linear Approximation of the Nonlinear Function 4 Gappy Approximation of the Nonlinear Function 5 Other Methods

David Amsallem & Charbel Farhat Stanford University cfarhat@stanford.edu

CME 345: MODEL REDUCTION - Nonlinear Systems

CME 345: MODEL REDUCTION - Nonlinear Systems Nonlinear Dynamical Systems General Framework

Full-Order Nonlinear Model: x y = f(x, u) = g(x, u).

x Rn : vector state variables u Rp : vector of input variables, typically p y Rq : vector output variables, typically q

n n

Usually, there is no closed form solution for x(t) = (t, u; t0 , x0 )

David Amsallem & Charbel Farhat Stanford University cfarhat@stanford.edu

CME 345: MODEL REDUCTION - Nonlinear Systems

CME 345: MODEL REDUCTION - Nonlinear Systems Nonlinear Dynamical Systems Model Reduction by Petrov-Galerkin Projection

Approximation of the state x(t) Vxr (t) Resulting nonlinear ODE Vxr = f(Vxr , u) + r(t) Enforce orthogonality of the residual r to a left basis W: WT Vxr = WT f(Vxr , u) If WT V is nonsingular xr = (WT V)1 WT f(Vxr , u)

David Amsallem & Charbel Farhat Stanford University cfarhat@stanford.edu

CME 345: MODEL REDUCTION - Nonlinear Systems

CME 345: MODEL REDUCTION - Nonlinear Systems Nonlinear Dynamical Systems Model Reduction by Petrov-Galerkin Projection

Reduced-order system of nonlinear ODEs xr = fr (xr , u) y = gr (xr , u)

vector of reduced coordinates: x Rk reduced-order nonlinear dynamical operator: fr : Rk Rp (xr , u) Rk (WT V)1 WT f(Vxr , u)

nonlinear output operator expressed in function of the reduced coordinates: gr : Rk Rp (xr , u)

David Amsallem & Charbel Farhat Stanford University cfarhat@stanford.edu

Rq g(Vxr , u)

CME 345: MODEL REDUCTION - Nonlinear Systems

CME 345: MODEL REDUCTION - Nonlinear Systems Nonlinear Dynamical Systems Solution by Explicit Time-Stepping

xr

(n)

denotes here an approximation to xr (t (n) )

Example: forward Euler scheme x(n+1) r y(n+1) = = = x(n) + (t (n+1) t (n) )fr (x(n) , u(t (n) )) r r x(n) + (t (n+1) t (n) )(WT V)1 WT f(Vx(n) , u(t (n) )) r r g(Vx(n+1) , u(t (n+1) )) r

Requires evaluating the functions f and g: cost scales with n

David Amsallem & Charbel Farhat Stanford University cfarhat@stanford.edu

CME 345: MODEL REDUCTION - Nonlinear Systems

CME 345: MODEL REDUCTION - Nonlinear Systems Nonlinear Dynamical Systems Solution by Implicit Time-Stepping

xr

(n)

denotes here an approximation to xr (t (n) )

Example: backward Euler scheme x(n+1) r y(n+1) = = = x(n) + (t (n+1) t (n) )fr (x(n+1) , u(t (n+1) )) r r
(n+1) x(n) + (t (n+1) t (n) )(WT V)1 WT f(Vxr , u(t (n+1) )) r

g(Vx(n+1) , u(t (n+1) )) r

Typically solved using Newton-Raphson method Requires evaluating the functions f and g: cost scales with n Requires evaluating the Jacobian of fr with respect to xr Ar (xr , u) where A(x, u) fr (xr , u) = (WT V)1 WT A(Vxr , u)V xr

f x (x, u)

Forming the reduced Jacobian can be extremely expensive!

David Amsallem & Charbel Farhat Stanford University cfarhat@stanford.edu CME 345: MODEL REDUCTION - Nonlinear Systems

CME 345: MODEL REDUCTION - Nonlinear Systems Linear Approximation of the Nonlinear Function Approach

Consider systems of the type x = f(x) + b(u) Linear systems are cheap as reduced-order operators of the type Ar = (WT V)1 WT AV can be formed once for all Idea: linearize f around an operating point x1 f(x) f(x1 ) + f (x1 )(x x1 ) = f(x1 ) + A(x1 )(x x1 ) x

The resulting system is then linear in the state x x = A(x1 )x + [b(u) + (f(x1 ) A(x1 )x1 )]
David Amsallem & Charbel Farhat Stanford University cfarhat@stanford.edu CME 345: MODEL REDUCTION - Nonlinear Systems

CME 345: MODEL REDUCTION - Nonlinear Systems Linear Approximation of the Nonlinear Function Model Reduction

Approximated full-order equations x = A(x1 )x + [b(u) + (f(x1 ) A(x1 )x1 )] Reduced-order equations xr = (WT V)1 WT A(x1 )Vxr +(WT V)1 WT [b(u) + (f(x1 ) A(x1 )x1 )] Ar = (WT V)1 WT A(x1 )V Rkk and Br = (WT V)1 WT [(f(x1 ) A(x1 )x1 )] Rk can be pre-computed

David Amsallem & Charbel Farhat Stanford University cfarhat@stanford.edu

CME 345: MODEL REDUCTION - Nonlinear Systems

CME 345: MODEL REDUCTION - Nonlinear Systems Piece-Wise Linear Approximation of the Nonlinear Function Approximation of the Nonlinear Function

Idea: Linearize the nonlinear function locally in the state space Approximated full-order dynamical system
s

x y

=
i=1

i (x)(f(xi ) + Ai (x xi )) + b(u)

= g(x, u)

i , i = 1, , s are weights such that

s

i (x) = 1, x D
i=1

David Amsallem & Charbel Farhat Stanford University cfarhat@stanford.edu

CME 345: MODEL REDUCTION - Nonlinear Systems

CME 345: MODEL REDUCTION - Nonlinear Systems Piece-Wise Linear Approximation of the Nonlinear Function Reduced-Order Modeling

Reduced-order model obtained after Petrov-Galerkin projection:

s

xr y with

=
i=1

i (xr )(WT V)1 (WT f(xi ) + WT Ai (Vxr xi )) + WT b(u)

= g(Vxr , u)
s

i (xr ) = 1, xr Dr .
i=1

Equivalently:
s s

xr

=
i=1

i (xr )Ari +br (u)

xr +
i=1

i (xr )(WT V)1 WT (f(xi ) Ai xi )

David Amsallem & Charbel Farhat Stanford University cfarhat@stanford.edu

CME 345: MODEL REDUCTION - Nonlinear Systems

CME 345: MODEL REDUCTION - Nonlinear Systems Piece-Wise Linear Approximation of the Nonlinear Function Reduced-Order Modeling

A complete model reduction method should provide algorithms for Selection of the linearization points {xi }s i=1 Selection of the reduced-order bases V and W Determination of the weights {i (xr )}s , xr Dr i=1

David Amsallem & Charbel Farhat Stanford University cfarhat@stanford.edu

CME 345: MODEL REDUCTION - Nonlinear Systems

CME 345: MODEL REDUCTION - Nonlinear Systems Piece-Wise Linear Approximation of the Nonlinear Function Selection of the Linearization Points

Linear approximations of the NL function only valid in a neighborhood of each xi . Cant cover the whole state-space D with local approximation Use trajectories (o-line phase) of the FOM to chose them Select linearization point if suciently far away from previous one Trajectory Piecewise Linear (TPWL) ROM (Rewisenski and White 2001).

David Amsallem & Charbel Farhat Stanford University cfarhat@stanford.edu

CME 345: MODEL REDUCTION - Nonlinear Systems

CME 345: MODEL REDUCTION - Nonlinear Systems Piece-Wise Linear Approximation of the Nonlinear Function Selection of the Right Reduced-Order Basis

Possible methods include If the input function is linear in u, construction of Krylov subspaces Ki = K(A1 , A1 B) = span(Vi ) at each linearization point xi and i i assembly of a global basis (Rewisenski and White 2001, Bond and Daniel 2004) V = [V1 , , Vs ] Ad-hoc methods (Balanced truncation, POD...)

David Amsallem & Charbel Farhat Stanford University cfarhat@stanford.edu

CME 345: MODEL REDUCTION - Nonlinear Systems

CME 345: MODEL REDUCTION - Nonlinear Systems Piece-Wise Linear Approximation of the Nonlinear Function Determination of the weights {i }

Used in the interpolation of the local reduced-order models to characterize the distance of the new point xr to the precomputed ones {xr ,i }s . i=1 Example (Bond and Daniel 2004): exp m2i
s j=1 d 2 d 2

i (xr ) =

exp m2j
2

where is a constant, di = xr xr ,i

and m = mins dj . j=1

Other expressions suggested in Tiwary and Rutenbar (2005), Dong and Roychowdhury (2005).

David Amsallem & Charbel Farhat Stanford University cfarhat@stanford.edu

CME 345: MODEL REDUCTION - Nonlinear Systems

CME 345: MODEL REDUCTION - Nonlinear Systems Piece-Wise Linear Approximation of the Nonlinear Function Properties

From Rewienski and White (2006) A posteriori error estimator available when f is negative monotone. Stability guarantee under assumptions on f and the choice of V and the weights { i (x)}s . i=1 Passivity preservation under similar assumptions.

David Amsallem & Charbel Farhat Stanford University cfarhat@stanford.edu

CME 345: MODEL REDUCTION - Nonlinear Systems

CME 345: MODEL REDUCTION - Nonlinear Systems Piece-Wise Linear Approximation of the Nonlinear Function Analysis of the Method

Strengths Does not scale with the size of the FOM n

Weaknesses

David Amsallem & Charbel Farhat Stanford University cfarhat@stanford.edu

CME 345: MODEL REDUCTION - Nonlinear Systems

CME 345: MODEL REDUCTION - Nonlinear Systems Piece-Wise Linear Approximation of the Nonlinear Function Analysis of the Method

Strengths Does not scale with the size of the FOM n

Weaknesses Choice of trajectories of the FOM is essential

David Amsallem & Charbel Farhat Stanford University cfarhat@stanford.edu

CME 345: MODEL REDUCTION - Nonlinear Systems

CME 345: MODEL REDUCTION - Nonlinear Systems Piece-Wise Linear Approximation of the Nonlinear Function Analysis of the Method

Strengths Does not scale with the size of the FOM n Global basis

Weaknesses Choice of trajectories of the FOM is essential Global basis

David Amsallem & Charbel Farhat Stanford University cfarhat@stanford.edu

CME 345: MODEL REDUCTION - Nonlinear Systems

CME 345: MODEL REDUCTION - Nonlinear Systems Piece-Wise Linear Approximation of the Nonlinear Function Analysis of the Method

Strengths Does not scale with the size of the FOM n Global basis

Weaknesses Choice of trajectories of the FOM is essential Global basis

David Amsallem & Charbel Farhat Stanford University cfarhat@stanford.edu

CME 345: MODEL REDUCTION - Nonlinear Systems

CME 345: MODEL REDUCTION - Nonlinear Systems Piece-Wise Linear Approximation of the Nonlinear Function Analysis of the Method

Strengths Does not scale with the size of the FOM n Global basis Availability of Jacobians

Weaknesses Choice of trajectories of the FOM is essential Global basis Availability of Jacobians

David Amsallem & Charbel Farhat Stanford University cfarhat@stanford.edu

CME 345: MODEL REDUCTION - Nonlinear Systems

CME 345: MODEL REDUCTION - Nonlinear Systems Piece-Wise Linear Approximation of the Nonlinear Function Analysis of the Method

Strengths Does not scale with the size of the FOM n Global basis Availability of Jacobians Not too much code intrusive

Weaknesses Choice of trajectories of the FOM is essential Global basis Availability of Jacobians

David Amsallem & Charbel Farhat Stanford University cfarhat@stanford.edu

CME 345: MODEL REDUCTION - Nonlinear Systems

CME 345: MODEL REDUCTION - Nonlinear Systems Piece-Wise Linear Approximation of the Nonlinear Function Analysis of the Method

Strengths Does not scale with the size of the FOM n Global basis Availability of Jacobians Not too much code intrusive

Weaknesses Choice of trajectories of the FOM is essential Global basis Availability of Jacobians Parameters to adjust

David Amsallem & Charbel Farhat Stanford University cfarhat@stanford.edu

CME 345: MODEL REDUCTION - Nonlinear Systems

CME 345: MODEL REDUCTION - Nonlinear Systems Piece-Wise Linear Approximation of the Nonlinear Function Extensions of TPWL

Piecewise Polynomial Representations (Roychowdhury 1999, Chen 1999, Philips 2003, Li and Pileggi 2003) Kernel-based Representations (Philips et al. 2003) Parametric Nonlinear ROMs (Bond and Daniel 2004) Localized Linear Reductions (Local bases) (Tiwary and Rutenbar 2006) ManiMOR (Gu and Roychowdhury 2008)

David Amsallem & Charbel Farhat Stanford University cfarhat@stanford.edu

CME 345: MODEL REDUCTION - Nonlinear Systems

CME 345: MODEL REDUCTION - Nonlinear Systems Gappy Approximation of the Nonlinear Function Gappy Reconstruction

First applied to face recognition (Emerson and Sirovich, Karhunen-Loeve Procedure for Gappy Data 1996)

David Amsallem & Charbel Farhat Stanford University cfarhat@stanford.edu

CME 345: MODEL REDUCTION - Nonlinear Systems

CME 345: MODEL REDUCTION - Nonlinear Systems Gappy Approximation of the Nonlinear Function Approach

Approximation of the nonlinear function f in x = f(x) + b(u) The evaluation of all the entries in the vector f() is expensive (scales with n) Only a small subset of these entries will be evaluated The other entries will be reconstructed either by interpolation or a least-squares strategy using a pre-computed reduced-order basis The state-space is still reduced by any preferred model reduction method (by POD for instance)

David Amsallem & Charbel Farhat Stanford University cfarhat@stanford.edu

CME 345: MODEL REDUCTION - Nonlinear Systems

CME 345: MODEL REDUCTION - Nonlinear Systems Gappy Approximation of the Nonlinear Function Approach

A complete model reduction method should then provide algorithms for Selecting the evaluation indices I = {i1 , , iNi } Selecting a reduced-order bases Vf for the nonlinear function Reconstructing the complete approximated nonlinear function vector f()

David Amsallem & Charbel Farhat Stanford University cfarhat@stanford.edu

CME 345: MODEL REDUCTION - Nonlinear Systems

CME 345: MODEL REDUCTION - Nonlinear Systems Gappy Approximation of the Nonlinear Function Computation of the Reduced-Order Basis for f()

A POD basis for f() is built:

1

Snapshots for the nonlinear function are collected from a transient simulation F = [f(x(t1 )), , f(x(tNs )] A singular value decomposition is computed F = Uf f ZT f

The basis is truncated and the kf rst vectors retained Vf = Ukf

David Amsallem & Charbel Farhat Stanford University cfarhat@stanford.edu

CME 345: MODEL REDUCTION - Nonlinear Systems

CME 345: MODEL REDUCTION - Nonlinear Systems Gappy Approximation of the Nonlinear Function Reconstruction of an Approximated Nonlinear Function

Assume Ni indices have been chosen I = {i1 , , iNi } The choice of indices will be specied later Consider the n-by-Ni matrix P = e i 1 , , e i Ni At each time t, for a given value of the state x(t) = Vxr (t), the entries in the function f corresponding to those indices will be evaluated fi1 (x(t)) . . PT f(x(t)) = . fiNi (x(t)) This is cheap if Ni n Usually only a subset of the entries in x(t) will be used to construct that vector
David Amsallem & Charbel Farhat Stanford University cfarhat@stanford.edu CME 345: MODEL REDUCTION - Nonlinear Systems

CME 345: MODEL REDUCTION - Nonlinear Systems Gappy Approximation of the Nonlinear Function Reduced-Order Dynamical System

Case where Ni = kf : interpolation

Idea: fij (x) = fij (x), j = 1, , Ni This means that PT f(x(t)) = PT f(x(t)) Remember that belongs to the span of the vectors in Vf , that is f() = Vf fr () f() Then PT Vf fr () = PT f(x(t)) Assuming PT Vf is nonsingular fr () = (PT Vf )1 PT f(x(t)) In terms of f(): = Vf (PT Vf )1 PT f(x(t)) = V ,P f(x(t)) f() f This results in an oblique projection of the full nonlinear vector
David Amsallem & Charbel Farhat Stanford University cfarhat@stanford.edu CME 345: MODEL REDUCTION - Nonlinear Systems

CME 345: MODEL REDUCTION - Nonlinear Systems Gappy Approximation of the Nonlinear Function Reduced-Order Dynamical System

Case where Ni > kf : least-squares reconstruction

Idea: fij (x) fij (x), j = 1, , Ni in the least squares sense Idea: minimize min PT Vf fr () PT f(x(t)) 2
fr ()

Notice that M = P Vf RNi kf is a skinny matrix One can compute its singular value decomposition
T

M = UZT The left inverse of M is then dened as M = Z UT

1 1 where = diag( 1 , , r , 0, , 0) if = diag(1 , , r , 0, , 0) with 1 r > 0 Then = Vf (PT Vf ) PT f(x(t)) f()

David Amsallem & Charbel Farhat Stanford University cfarhat@stanford.edu

CME 345: MODEL REDUCTION - Nonlinear Systems

CME 345: MODEL REDUCTION - Nonlinear Systems Gappy Approximation of the Nonlinear Function Selection of the Sampling Indices

f f This selection takes place after the vectors [v1 , , vkf ] has been computed by POD

Greedy algorithm (Chaturantabut 2010):

1: 2: 3: 4: 5: 6: 7: 8:
f [m, i1 ] = max{|v1 |} f Vf = [v1 ], P = [ei1 ] for l = 2 : kf do Solve PT Vf c = PT vlf for c r = vlf Vf c [m, il ] = max{|r|} Vf = [Vf , vlf ], P = [P, eil ] end for

David Amsallem & Charbel Farhat Stanford University cfarhat@stanford.edu

CME 345: MODEL REDUCTION - Nonlinear Systems

CME 345: MODEL REDUCTION - Nonlinear Systems Gappy Approximation of the Nonlinear Function Application

The inviscid Burgers equation: U(x, t) 1 (U(x, t))2 + = g (x) t 2 x where the source term is g (x) = 0.02exp(0.02x) the initial condition is U(x, 0) = 1 and the inlet boundary condition is U(0, t) = 5

Discretized by a Finite Volume Method (Godunov)

David Amsallem & Charbel Farhat Stanford University cfarhat@stanford.edu

CME 345: MODEL REDUCTION - Nonlinear Systems

CME 345: MODEL REDUCTION - Nonlinear Systems Gappy Approximation of the Nonlinear Function Application

Results of the greedy algorithm

Inde x Greedy Selection
1 0.8 0.6 0.4 0.2

13106 42227 20923 258 7 24 112814 16 21 3 1718 129 26 15 5 30 12 19 0

0.2 0.4 0.6 0.8 1 0 20 40 60 80

100

120

David Amsallem & Charbel Farhat Stanford University cfarhat@stanford.edu

CME 345: MODEL REDUCTION - Nonlinear Systems

CME 345: MODEL REDUCTION - Nonlinear Systems Gappy Approximation of the Nonlinear Function Application

k = 15, kf = 40, Ni = 40
4.5 4 3.5 3

U
2.5 2 1.5 1 0

20

40

60

80

100

Similar results with Ni > 40 (least squares reconstruction)

David Amsallem & Charbel Farhat Stanford University cfarhat@stanford.edu

CME 345: MODEL REDUCTION - Nonlinear Systems

CME 345: MODEL REDUCTION - Nonlinear Systems Gappy Approximation of the Nonlinear Function Application

k = 15, kf = 30, Ni = 80
5

1 0

20

40

60

80

100

Similar results with Ni = 100 (no gaps) kf is too small

David Amsallem & Charbel Farhat Stanford University cfarhat@stanford.edu CME 345: MODEL REDUCTION - Nonlinear Systems

CME 345: MODEL REDUCTION - Nonlinear Systems Gappy Approximation of the Nonlinear Function Application

Method Full-Order Model ROM (Galerkin) ROM (Gappy approximation)

CPU Time 2180 s 1866 s 183 s

Relative Error 1.95% 2.17%

Table: Comparison of CPU Timings and Accuracy of the Methods

David Amsallem & Charbel Farhat Stanford University cfarhat@stanford.edu

CME 345: MODEL REDUCTION - Nonlinear Systems

CME 345: MODEL REDUCTION - Nonlinear Systems Other Methods Incomplete List

Balanced Truncation for NL systems Volterra series Methods for Linear Time Variant and Periodic Systems

David Amsallem & Charbel Farhat Stanford University cfarhat@stanford.edu

CME 345: MODEL REDUCTION - Nonlinear Systems