Advanced Topics in Control Systems

Exercises and Project Ideas

Pantelis Sopasakis
KU Leuven, Department of Electrical Engineering (ESAT), STADIUS Center for Dynamical
Systems, Signal Processing and Data Analytics, Kasteelpark Arenberg 10, 3001 Leuven,
Belgium.

Abstract
For the final evaluation for the PhD course Advanced Topics in Con-
trol Systems you either have to do a project related to the course, or solve
a set of exercises. In this document you will find a few ideas for projects
and the exercises which you, alternatively, have to solve. Address any
questions to Pantelis Sopasakis.

Contents
1 Project ideas 2
1.1 Literature review . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2 Application projects . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 Theoretical studies . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.4 Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2 Exercises 4
2.1 Economic MPC . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Dissipativity and economic MPC (8). . . . . . . . . . . . . . . . 5
2.2 Markovian processes and systems . . . . . . . . . . . . . . . . . . 5
Markovianity: History does not matter (3). . . . . . . . . . . . . 5
MJLS oddity (2+2). . . . . . . . . . . . . . . . . . . . . . . . . . 6
Stabilization of MJLS (2+3). . . . . . . . . . . . . . . . . . . . . 6
Limit probability (3). . . . . . . . . . . . . . . . . . . . . . . . . 6
Stochastic MPC (10). . . . . . . . . . . . . . . . . . . . . . . . . 6
MSS sequence with diverging higher moment (2). . . . . . . . . 7
Diverging third moment in MJLS (6). . . . . . . . . . . . . . . . 7
2.3 Stochastics and Risk measures . . . . . . . . . . . . . . . . . . . 7
Average value-at-risk of normally distributed variables (2). . . . 7
Markov decision processes and risk-averse decision making (4). . 7
New risk measures from old ones (2). . . . . . . . . . . . . . . . 8
Coherency (4). . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

1
 Zero Risk (5). . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Subgradient (7). . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Polytopic risk measure (5). . . . . . . . . . . . . . . . . . . . . . 9
Strict monotonicity of AV@R (5). . . . . . . . . . . . . . . . . 9
Strict monotonicity (5). . . . . . . . . . . . . . . . . . . . . . . . 9
Strong monotonicity (7). . . . . . . . . . . . . . . . . . . . . . . 9
Dual representation of certain risk measures (3+8). . . . . . . . 9
AV@R of sum (3). . . . . . . . . . . . . . . . . . . . . . . . . . 10
Conditional risk mappings (7). . . . . . . . . . . . . . . . . . . . 10
Conditional risk mappings from coherent risk measures (3). . . . 10
Coherent risk measures from conditional risk mappings (3). . . . 10
Equivalence of optimization problems (5). . . . . . . . . . . . . . 11
Separable expected conditionals (5). . . . . . . . . . . . . . . . . 11
Different random variables, same distribution (5). . . . . . . . . 11
Information monotonicity of multiperiod risk measures (5). . . . 11

1 Project ideas
Any project related to the course is acceptable. It can be either a literature
survey, or a computational project, or a theoretical study. In either case, you
have to prepare a report with the structure of a conference or journal article, 6
to 9 pages long and up to 12 pages for literature surveys in the IFAC conference
double-column format.

1.1 Literature review

If you choose the literature survey, you can look into the following topics:1

Economic MPC
i. Economic MPC and relevant applications, ii. Economic vs conventional RTO/
MPC methodologies, iii. Open problems and future directions in economic MPC.

Markovian and stochastic systems

i. MPC for constrained stochastic systems, ii. Scenario - based stochastic MPC
and applications (e.g., supply chains, portfolio selection, robotics, energy man-
agement and dispatch, water networks, automotive applications, etc), iii. Sce-
nario generation algorithms, iv. Chance-constrained MPC, v. Almost sure sta-
bilisation of stochastic systems, vi. Stochastic tube methods, vii. Open problems
and future research directions in MPC for stochastic systems.
1 The topics listed here are only indicative. Feel free to study any different problem which

is related to the course.

2
Risk measures
i. Risk measures and their properties (strict/strong monotonicity and other
theoretical properties) several novel risk measures have been proposed in the
literature such as the expectiles. A survey on risk measures would attempt to
summarise the merits of the various risk measures found in the literature and
discuss their potential applications, ii. Statistical estimation of risk measures,
iii. The Kusuoka theorem and its applications.

Risk-averse optimisation and control

i. Risk-averse vs risk-neutral problem formulations, ii. Two-state vs multistage
risk-averse formulations: differences, properties, numerical methods, iii. Dual-
isation of non-anticipativity constraints and applications, iv. Risk-averse opti-
mal control and distrubutionally robust problem formulations, v. The problem
of moments in risk-averse optimisation, vi. New formulations of optimal control
problems involving risk measures and their properties such as formulations based
on separable expected conditionals, vii. Information monotonicity of risk-averse
formulations, viii. Discussion of the property of time consistency of multistage
risk-averse problems.;

1.2 Application projects

Application projects can showcase the application of (i) economic MPC or (ii)
stochastic MPC or (iii) risk-averse optimal control for the solution of a practical
problem. You need to provide a short literature background indicating the
state of the art, introduce the theoretical tools you used and demonstrate with
simulations the advantages of the solution you propose. You may also combine
any of the ideas listed above with simulations.

1.3 Theoretical studies

You may address one or more theoretical questions in any of the above fields.

1.4 Requirements
Exercises. Your solutions to these exercises have to be typeset in LATEX(or
similar typesetting programmes like LuaTeX, XeTeX, etc). Acceptable solutions
must be rigorous, explain all involved steps and state clearly all assumptions
these will be the criteria for their evaluation. Figures must include proper axis
titles and captions and they need to be properly typeset (do not use screenshots).
Projects. Similarly, all projects must be typeset in LATEXand have an ab-
stract and a Conclusions section. If your project is a literature review, it will
be graded on the basis of how complete it is and primarily how good
understanding of the literature you have.

3
 Submission. Send your final submission by email to Pantelis Sopasakis CC-
ing Prof. Alberto Bemporad with subject [ATCS] Final project or [ATCS]
Exercises.

2 Exercises
This is a set of 27 exercises on three thematic sections: economic model predic-
tive control, Markovian stochastic processes and systems and risk measures. In
each exercise, its contribution to the total mark is shown in parentheses. The
maximum mark is 100, while all marks sum up to 134. This allows you to choose
which exercises you will solve. Unless otherwise stated, subquestions give equal
points. These exercises are an alternative to doing your own project.

Definitions and Preliminaries

In what follows, unless otherwise specified, (, F , P) is a probability space, 6=
, Z = Lp (, F , P) with p [1, ), Z = Lq (, F , P) is the topological dual of
Z with q such that q 1 +p1 = 1. For X Z and R Y Z we

R define the product
of the two random variables to be hX, Y i = XY dP = X()Y ()dP().
The abbreviation a.s. stands for almost surely. The characteristic function of a
set A is defined as (
1, if x A,
1A (x) =
0, otherwise
The average value-at-risk of Z Z is defined as

AV@R [Z] = inf {t + 1 E[Z t]+ },

tR

where [ Z() ]+ = max{Z(), 0}. The essential supremum of a random variable

Z on (, F , P) is defined as

ess sup Z = inf {P[Z > ] = 0}.

R

A risk measure is called coherent if it satisfies the following assumptions for

Z, V Z, 0 and a R
1. Subadditivity (Z + V ) (Z) + (V )
2. Positive homog. (Z) (Z)
3. Monotonicity Z V (Z) (V )

4. Translation invariance (Z + a) = a + (Z)

We say that two random variables X and Y on (, F , P) are equal in dis-
tribution if for all A F , P[X A] = P[Y A]. This does not imply that the
two random variables are P-a.s. equal.

4
 Let Zi = Lp (, Fi , P) for i = 0, . . . , N be a sequence of probability spaces
with ZN = Z (FN = F ) and F0 = {, } is the trivial -algebra. If a random
variable Z is Z0 -measurable, then, in fact, it is deterministic. We say that a
mapping : Z0 Z1 . . . ZN is a multiperiod risk measure if it satisfies
the following axioms for Z = (Z0 , Z1 , . . . , ZN ) and Z 0 = (Z00 , Z10 , . . . , ZN
0
) in
Z0 Z 1 . . . Z N
1. Subadditivity (Z + Z 0 ) (Z) + (Z 0 )
2. Positive homog. (Z) (Z)
3. Monotonicity Z Z 0 (Z) (Z 0 )
4. Translation invariance: for every random variable Y which is Fi -measurable,
i = 0, . . . , N 1, it is (Z0 , . . . , Zk , Zk+1 + Y, . . . , ZN ) = (Z0 , . . . , Zk +
Y, Zk+1 , . . . , ZN ).
Notice that the first three axioms are very similar to the ones stated above for
coherent risk measures.

2.1 Economic MPC

Max. points in this section: 8.

11. Dissipativity and economic MPC (8). Consider the following non-
linear discrete-time system with 2 states and 1 input

xk+1 = xk + (1 + kxk k)uk

and the cost functional ` : Rn R R given by

`(x, u) = kxk (|1 u| 21 ).

The system is subject to the constraints xk [5, 5] [5, 5] and uk [2, 2].
The norm k k is the Euclidean norm. (i) Find all equilibrium points of the
system, (ii) verify that xs = 0, us = 0 is an optimal steady state with respect to `
and show that the system is dissipative with respect to the supply rate s(x, u) =
`(x, u) `(xs , us ), (iv) implement an economic model predictive controller using
` as a stage cost, with horizon N = 15 and using the terminal constraint xN = 0.
Demonstrate with simulations that your formulation is recursively feasible.

2.2 Markovian processes and systems

Max. points in this section: 33.

21. Markovianity: History does not matter (3). Does it? Often, a
rough description of what a Markov process runs
a process where the past (times k 1, k 2, . . .) has no effect on
the future (time k + 1) once the present is known (time k).

5
This said, one would infer that the following is correct

P[xk+1 A | xk B, xk1 C] = P[xk+1 A | xk B],

for all k N and for measurable sets A, B and C. Although Fukuyama would
probably find this correct, it is not. Can you see why? Show this with a
counterexample.

22. MJLS oddity (2+2). Construct a Markovian system xk+1 = Ak xk

with xk R2 so that Ak are all stable matrices, E[xk ] 0, but the covariance
of xk diverges.
Construct an MJLS xk+1 = Ak xk + Bk uk with k {1, 2} so that (A1 , B1 )
is not stabilisable, but there are Ki so that xk+1 = (Ak + Bk Kk )xk is mean
square stable.
Find an example of a system with xk+1 = Ak xk with xk R2 and k
{1, 2} which is almost surely convergent, but not mean-stable, that is E[xk ]
diverges.

23. Stabilization of MJLS (2+3). Consider a MJLS with two modes and
       
0 1 1 1 1 1
A1 = , A2 = , B1 = , B2 = .
1 0 1 0 0.1 0

(i) Design a linear feedback u(x, i) = Ki x which stabilises the closed-loop system
in the mean-square sense. (ii) Assume that k is not known at time k, but,
instead, an estimate k of it is available. We know that

P[ = 1 | = 1] = 0.97
P[ = 2 | = 2] = 0.95.

Design a feedback controller of the form u(x, ) = K x so that the closed-loop

system, that is

xk+1 = (Ak + Bk Kk )xk ,

is mean-square stable. Provide simulation results.

24. Limit probability (3). Consider a Markov chain with 3 modes and
transition matrix
0.1 0.2 0.8
P = 0.2 0.1 0.7
0.9 0 0.1
(i) Compute its limit probability. (ii) Assume that the initial distribution is
v = [ 0.2 0.1 0.7 ] that is, P[0 = 1] = 0.2, P[0 = 2] = 0.1 and P[0 = 2] = 0.7.
Determine the probabilities P[5 = i], P[20 = i], P[50 = i] for i = 1, 2, 3.

25. Stochastic MPC (10). Consider a Markovian system describing an

industrial process with two modes corresponding to proper (1) and faulty (2)

6
operation. The transition matrix between the two modes is
 
0.98 0.01
P =
0.1 0.9

The system data are

     
1.0 0.7 1.1 0.9 0.8
A1 = , A2 = , B1 = B2 = .
0.75 0.4 0.9 0.3 0.1

The system is subject to the state and input constraints

   
5 5
xk
5 5
3 uk 3

Design and implement (e.g., in MATLAB) a stochastic model predictive con-

troller with horizon N so that the controlled system satisfies the constraints
and renders the closed-loop system mean-square stable. Simulate the closed-
loop behaviour of the system starting from various initial states.

26. MSS sequence with diverging higher moment (2). Take a random
variable np with parameter p which takes values in = {0, n}, with probability
distribution P[np = n] = p n. Let X = 0 and Xk , k N, be defined as Xk = kpk
where pk = 1/(k 2 ln k). Show that Xk converges to 0 in the mean-square sense,
but E[|Xk |s ] diverges for all s > 2.

27. Diverging third moment in MJLS (6). Consider a Markovian linear

system xk+1 = Ak xk , where k is a Markov process. Suppose that the system
is mean-square stable. Does kxk kp always converge for p (2, )?

2.3 Stochastics and Risk measures

Max. points in this section: 93.

31. Average value-at-risk of normally distributed variables (2). Take

a normally distributed real-valued random variable Z N (, 2 ). Show that

1 (1 )
 

AV@R [Z] = + exp
2 2

32. Markov decision processes and risk-averse decision making (4).

Consider the problem shown in the following figure which shows a Markov de-
cision process. This is an one-player game which works as follows: assume we
are at node A. We have two possible actions: a and b. If we choose b we know
that the next node will be B; this decision will cost us e10. Otherwise, if we
choose to make the decision a we will either land in node A and pay e100, or

7
land in B and pay e2. Clearly, not going for the deterministic choice b incurs a
risk: we will either pay less (with probability 1 ), or, we risk to have to pay
e100 (with probability ). When at node B, we are facing an even more extreme
high-gain-high-loss situation: we will either choose to pay e10 by choosing b,
or choose a and risk to pay e800 with probability 1 .

B
b a
a
b

Let  = 0.01. How would a risk-neutral decision maker (who decides using E)
behave? How would a decision maker behave if they account for the worst-case
outcome? How would one play if they used the average value-at-risk with some
confidence level (0, 1)?

33. New risk measures from old ones (2). Let : Z R be a coherent
risk measure. Then, show that = E + (1 ), with [0, 1], is a coherent
risk measure too.

34. Coherency (4). Show that the following functional

log E(eX )
H (X) = ,

which is known as the exponential premium, in not a coherent risk measure.

35. Zero Risk (5). Let (, F , P) be a nonatomic probability space and

: Z R be a law-invariant risk measure. Assume that is proper, lower
semicontinuous and coherent. Let Z be an almost-everywhere nonnegative ran-
dom variable on (, F , P). Then, prove that [Z] = 0 if and only if Z = 0
almost everywhere.

36. Subgradient (7). Let : Z R be a lower semicontinuous coherent

risk measure. The subgradient of at X0 Z is defined as
  
 (X) (X0 ) hY, X X0 i
()[X0 ] = Y Z 
for all X Z

8
Given that is given as [Z] = supY A hY, Zi, show that

()[X0 ] = arg maxhY, X0 i.

Y Z

37. Polytopic risk measure (5). Let be a polytopic risk measure, that
is, let Y1 , . . . , Y Z with Yi 0 a.s. and E[Y ] = 1 and let

A = conv{Yi }i=1,..,

that is, A is the convex hull of Y1 , . . . , Y . Define a risk measure : Z R

by [Z] = maxY A hY, Zi. Show that is a coherent risk measure, compute
its subgradient and show that for all Z Z , (Z) is nonempty, convex and
bounded.

38. Strict monotonicity of AV@R (5). We know that AV@R is mono-

tone in the sense that AV@R [Z1 ] AV@R [Z2 ] whenever Z1 Z2 . As-
sume (0, 1). Then show that AV@R is also strictly monotone, i.e.,
AV@R [Z1 ] < AV@R [Z2 ] whenever Z1 < Z2 . Show with a counterexample
that the essential supremum operator does not possess this property.

39. Strict monotonicity (5). Let be strictly monotone. Show that

= E + (1 ), with [0, 1], is also strictly monotone.

310. Strong monotonicity (7). Let : Z R be a coherent risk measure.

For Z, Z 0 Z we denote Z Z 0 whenever Z() Z 0 () for almost all
and P(Z < Z 0 ) > 0. We say that is strongly monotone if Z Z 0 implies
that (Z) < (Z 0 ). Show that (i) The expectation E is strongly monotone, (ii)
AV@R is not strongly monotone (you can do that by counterexample), (iii) if
is strictly monotone and (0, 1], then = E + (1 ) is strongly monotone,
(iv) is strongly monotone if and only if for all Z Z and for all Y (Z) it
is Y > 0 a.e.

311. Dual representation of certain risk measures (3+8). Recall that

the p-norm (the norm of Lp (, F , P)) is defined as
Z 1/p
kZkp := Z p dP

for p [1, ) and

kZk = ess sup [ Z ],
where ess supZ is the essential supremum of Z. Show that (i) p : Lp (, F , P)
R defined by p [Z] = kZkp are coherent risk measures; (ii) show that for
p [1, ) p is strongly monotone; (iii) show that for p [1, ) derive the
dual representation of p , that is, find sets Qp Lq (, F , P) so that

p [Z] = sup h, Zi.

Qp

9
(iv*)(optional, 8 points) Write in the form
[Z] = sup E [Z].
Q

Hint: For (iv), keep in mind that the dual of L (, F , P) is not an Lp space,
but the space ba(, F , P) the space of bounded and finitely additive signed
measures on (, F ) which are absolutely continuous with respect to P.

312. AV@R of sum (3). Let Z = Z0 . . . ZN 1 and % : Z R be

defined as follows
"N 1 #
X
%[Z] = %[Z0 , Z1 , . . . , ZN 1 ] = AV@R Zi ,
i=0

for Z Z (Zi Zi for i = 0, . . . , N 1). However, this risk measure is not

used in practice in the contect of multistage risk-averse optimisation because it
suffers from certain pathodologies. Can you spot them?

313. Conditional risk mappings (7). Consider two spaces of random

variables Zi = Lp (, Fi , P), i = 1, 2 with F1 F2 . Let us recall the following
definition: A mapping : Z2 Z1 is called a conditional risk mapping if it
satisfies the following axioms
1. (Z + (1 )Z 0 ) (Z) + (1 )(Z 0 ) for all [0, 1] and Z, Z 0 Z2
2. (Z) (Z 0 ) for all Z, Z 0 Z2 with Z Z 0 .
3. For all Y Z1 and Z Z2 , (Y + Z) = Y + (Z)
4. For every 0 and Z Z2 , (Z) = (Z).
where are meant in the almost-sure sense. Show that for Y Z1 , Y 0
(a.s.) it is (Y Z) = Y (Z).

314. Conditional risk mappings from coherent risk measures (3). Let
Zi = Lp (, Fi , P) with F1 F2 . Given a coherent risk measure : Z2 R
which has the dual representation
(Z) = sup E [Z],
A

let us define the mapping F1 : Z2 Z1 as

F1 (Z) = sup E [Z | F1 ].
A

Show that F1 is a conditional risk mapping.

315. Coherent risk measures from conditional risk mappings (3). Let
Zi = Lp (, Fi , P) with F1 F2 . Given a conditional risk mapping : Z2
Z1 and a set A F1 with P(A) > 0 define the mapping A
A (Z) = h1A , (Z)i/P(A).

10
Show that A is a coherent risk measure. Extend the above result by showing
that for a F1 -measurable random variable Z1 with dP 6= 0 and 0
R

a.s., the mapping R

(Z)dP
(Z) = R

dP
is a coherent risk measure.

316. Equivalence of optimization problems (5). Here let Z 0 = Lp (, F , P)

with F 0 F . Let f : Z R, C : Z Z 0 and g : Z 0 R be convex mappings.
The mapping C is convex in the sense C(Z +(1)Z 0 ) C(Z)+(1)C(Z 0 )
P-a.s. for all [0, 1]. Consider the optimisation problem

P : minimise f (X) + g(C(X))

XZ

and the optimisation problem

P0 : minimise {f (X) + g(R); C(X) R}.

XZ
RZ 0

Let (X ? , R? ) be an optimiser of P0 . (i) Show that X ? is an optimiser of P. (ii)

Assume now that X ? and optimiser of P and assume that it is unique. Show
that there is an R? Z 0 so that (X ? , R? ) is an optimiser of P0 , there may be
an R?? 6= R? so that (X ? , R?? ) is a minimiser of P. (iii) Under what conditions
is R? unique?

317. Separable expected conditionals (5). Take the mapping % : Z0

Z1 . . . ZN R defined as
N
X
%[Z0 , Z1 , . . . , ZN 1 ] = Z0 + (Z1 ) + E[(Zk | Fk1 )],
k=2

where ( | Fk1 ) : Zk Zk1 , for k = 2, . . . , N , is a conditional risk mapping

(derived by some coherent risk measure ). Show that (i) this is a multiperiod
risk measure and (ii) there is a coherent risk measure : ZN 1 R so that
%(Z0 , . . . , ZN 1 ) = (Z0 + . . . + ZN 1 ).

318. Different random variables, same distribution (5). Let = {0, 1}

be a finite sample space with two events. Give an example of two random
variables X and Y on a probability space (, 2 , P) for some P which are
equal in distribution, but never equal (that is X() 6= Y ()).
Let = R equipped with the Borel -algebra. Give an example of two
random variables X and Y on (, BR , P) for some probability measure P
which are equal in distribution, but almost surely not equal (that is X() 6=
Y () P-a.s.).

319. Information monotonicity of multiperiod risk measures (5). We

say that a mutliperiod risk measure : Z0 , . . . , ZN R is information-monotone

11
if for two filtrations F = {Fk }k=0,...,N and G = {Gk }k=0,...,N with Fk Gk for
all k = 0, . . . , N , it is2

(Z0 , . . . , ZN ; F) (Z0 , . . . , ZN ; G).

This means that more information (that is, G) cannot increase the total multi-
period risk3 .
Consider the following nested multiperiod risk functional

(Z0 , Z1 , . . . , ZN ) = Z0 + 1 (Z1 + . . . + N 1 (ZN 1 + N (ZN ))),

where Zk is Fk -measurable and k are conditional risk mappings.

Construct a simple example to show that such problems may not be
information-monotone in the sense of the definition given above4 .

N is F-adapted (i.e., Zk are Fk -measurable), then it is also G-adapted. The

2 If Z , . . . , Z
0
converse is not true. Here we assume that Z0 , . . . , ZN is F-adapted.
3 Recall that we proved in class that for law invariant, lower semicontinuous and coherent

risk measures, information does not increase risk (that is (E[Z | G ]) (Z) for any -algebra
G ).
4 Hints: Assume that have some simple form, e.g., they are conditional average value-at-
k
risk mappings. Take N = 2 for simplicity. Construct a simple scenario tree which corresponds
to a filtration F (with a bew branches at each stage). Introduce a finer filtration G (increase
the branching). Evaluate (Z0 , . . . , ZN ; F) and (Z0 , . . . , ZN ; G).

12