Professional Documents
Culture Documents
Hugo Hernandez
ForsChem Research, 050030 Medellin, Colombia
hugo.hernandez@forschem.org
doi: 10.13140/RG.2.2.23850.11204
Abstract
Randomness emerges as the result of missing information. The presence of random variables in
a model indicates that we are missing some critical information about the system. Such lack of
knowledge propagates through functions involving random variables, including their
derivatives. The purpose of this report is to show how to calculate the probability density
function for the derivatives of random variables. As a general observation, the classical rules
for the calculus of derivatives also apply to mathematical functions of random variables. The
resulting probability density function can then be found by using the change of variables
theorem. On the other hand, it is also possible to consider derivatives of independent variables.
In this case, the probability density function of the derivative of an independent variable will
always have a mean of zero, and it will be symmetrical around the mean. The multidimensional
randomness of a system, captured through statistical or randomistic models can also be
differentiated, and the probability density function of the derivative can be obtained
analytically. In general, when normal random variables are involved, the resulting probability
density function of the derivative can be approximated by a normal distribution.
Keywords
1. Introduction
In previous reports [1,2], it was shown that the derivatives of random variables are also random
variables. Therefore, derivatives of continuous random variables can also be described by
probability density functions. The purpose of this report is to present the mathematical
expressions for determining the probability density functions of derivatives of continuous
random variables, and discuss some particular results.
Regarding the derivatives of a certain random variable with respect to another random
variable , it is important to consider two cases that behave differently: 1) depends on , 2)
and are independent of each other. The first case will be discussed in Section 2, whereas the
second case in Section 3. Finally, probability density functions of multivariate functions of
random variables will be considered in Section 4.
Let us consider the following random variable described by the function of the random
variable [1]:
(2.1)
where is described by the probability density function . Thus, from the change of variable
theorem [3], the probability density function of will be given by:
∑ ( )| |
(2.2)
where denotes the -th solution, and is the total number of different possible
solutions of .
(2.3)
(2.4)
∑ ( )| |
(2.5)
Thus, as long as the derivative of the function has inverse, it is possible to determine the
probability density function of the derivative from the probability density function of using
the change of variable theorem.
Example 2.1.
In this case,
(2.6)
The function described by Eq. (2.6) has no inverse for obtaining , and therefore Eq. (2.5)
cannot be used. However, since the derivative is a constant, then its probability density
function is:
(2.7)
where represents Dirac’s delta function.
Example 2.2.
(2.8)
The inverse function is:
(2.9)
with only one solution. Thus, from Eq. (2.5):
( )
( )| | ( )
(2.10)
From Eq. (2.8), it can be concluded that the expected value of the derivative is: ,
and its variance is: .
( ) ( )
√ √
(2.11)
which corresponds to the probability density function of a normal random variable with mean
, and standard deviation .
(2.12)
For this example, the probability density function of is always the same function of , but
scaled to yield a distribution with twice the mean value, and twice its standard deviation.
Example 2.3.
In this case,
(2.13)
And the inverse function is:
(2.14)
with a single solution. From Eq. (2.5):
( )
( )| |
(2.15)
The expected value of the derivative is (from Eq. 2.13): , and its variance is:
. Different results will be obtained depending on the probability density
function of .
√
(2.16)
Figure 1. Probability density function of the derivative of (red solid line), compared to
the probability density function of a standard normal random variable (blue dashed line).
Example 2.4.
As a final example for this section, let us consider the function , with
derivative:
(2.17)
The corresponding inverse function is:
(2.18)
(2.19)
(2.20)
where represents the function yielding the principal value (in the range ) of
the inverse cosine function.
( )
( )| | ( )| |
( ) ( )
√
(2.21)
{ √
(2.22)
The resulting probability density function is depicted in Figure 2.
If is not restricted to the interval [ , for example, when it is a normal random variable
with mean , standard deviation , and , then since the range of such variable is
√
, all possible solutions must be considered in Eq. (2.5) yielding:
( ( ) ) ( ( ) )
∑ ( )
√ √
(2.23)
where . The probability density function obtained when is a standard random variable is
presented in Figure 3.
(3.1)
where is the random state of when is at state , and is the random state of when
is at state . The definition of the derivative presented in Eq. (3.1) implies that is forced to
be close to . Now, considering that [1]:
(3.2)
Then, Eq. (3.1) can be expressed as:
(3.3)
If the probability density function for and is for both, then the probability density
function of the derivative of can be determined by using the multivariate change of variable
theorem [3]:
∫ ( )
(3.4)
Thus, it is always possible to determine the probability density function for the derivative of an
independent random variable. Furthermore, there is always only one solution of Eq. (3.3),
which simplifies the calculations.
On the other hand, the expected value of the derivative is (from Eq. 3.1):
(3.5)
(3.6)
(3.7)
The main conclusion obtained from Eq. (3.7) is that for an independent random variable, the
variance of its derivative is inversely proportional to the square of the differential considered,
confirming a result that was previously reported for dynamical systems [2]. Thus, since
, . This result explains why for noisy signals, the noise of the
derivative is amplified unless certain mathematical filters are used.
If the value of is known with certainty, that is , then Eq. (3.3) becomes:
(3.8)
And the probability density function of is now:
( )
(3.9)
The expected value of the derivative in this case is:
(3.10)
and its variance is:
(3.11)
That is, when is known, the probability density function of the derivative is the same type or
family of the probability density function of .
Now, since in practice is small but not zero, it is possible to analyze the mathematical
behavior of the probability density function of the derivative for different independent random
variables.
Assuming that is a normal random variable with mean , standard deviation , and
probability density function:
√
(3.12)
Then, from Eq. (3.4) the behavior of the derivative when is unknown is:
( )
∫
√ √
( )
∫
√ √
√
( )
√
√ ( )
(3.13)
Eq. (3.13) represents a normal random variable with zero-mean and a standard deviation of
√
. This result is consistent with the values obtained by Eq. (3.5) and (3.6).
On the other hand, if is known, then from Eq. (3.9) the probability density function is:
( )
( ) ( )
√
√ ( )
(3.14)
which represents a normal random variable with a mean value of and standard
deviation , again consistent with the results obtained in Eq. (3.10) and (3.11).
Thus, from these results it can be concluded that the derivative of a normal random variable is
always a normal random variable. Furthermore, it can be inferred that all subsequent
derivatives of a normal random variable are also normal.
(3.15)
The probability density function of the derivative when is unknown is in this case:
( )
( )
{
( )
( )
{
| |( )
(3.16)
This result yields a triangular probability density function. An example of such probability
density function can be seen in Figure 4 considering , and .
On the other hand, if is known, then from Eq. (3.9) the probability density function is:
(3.17)
which is a uniform distribution, as expected.
For an independent exponential random variable, the probability density function is:
{
(3.18)
The probability density function of the derivative when is unknown is:
| |
{
(3.19)
( )
{
(3.20)
As a final example for this section, let us consider an independent standard Maxwell-Boltzmann
random variable with the following probability density function [4]:
(3.21)
§
When is unknown, the probability density function of the derivative is :
( ( ) )
( ) ∫ ( )
( )
( )
[ | | ( | |)
( )
√ (√ | |) ( ( ) ( ) )]
(3.22)
The expected value of the derivative is zero, and its variance is**:
(3.23)
The behavior of the probability density function can be seen in Figure 6, considering
. Interestingly, such probability density function can be approximated by a
§
Result obtained with the help of Wolfram-Alpha for some integrals.
**
Since for a standard Maxwell-Boltzmann random variable [4].
√
normal distribution with a zero-mean and a standard deviation of (obtained from Eq.
3.23), with a similitude [5] of .
Finally, for this example, if is known the probability density function is:
( )
( )
{
(3.23)
All the examples presented show that when the initial state is unknown, the derivative of an
independent random variable is a random variable with a probability density function that is
symmetrical around zero. In many cases, the resulting random variable can be approximated by
a normal distribution.
The most general case to be considered is the derivative of a multivariate function of random
variables. Any random variable considered in Nature will be affected by many different factors
simultaneously. However, it is usually impossible to consider all factors in single function, thus
giving rise to multidimensional randomness [6]. Such multidimensional random function can be
expressed as follows:
(4.1)
where is any function of known factors and unknown factors , which can be
approximated by a function of all known factors and the expected value of the unknown
factors , and a residual randomness described by a Type I standard random variable [6]
(mean and variance ), and a standard deviation parameter for .
[ [ [ [ ] is a vector representing all known factors. The expected values of
the unknown factors, which are also unknown, are present in the parameters of the function
.
Thus, the variable Y can be expressed as the following statistical (or randomistic††) model [7]:
(4.2)
Now, the total differential of the multivariate function at the random state
[ [ [
{ } is [1]:
[
∑ ∑( [
) [ [
( )
[ [ [
(4.3)
††
The term randomistic is used here to represent the fact that the model is based on random instead of
deterministic variables. The term randomistic is considered by the author of this report a better
description of the nature of the model compared to the term statistical.
Eq. (4.3) already considers a finite difference correction in the true state of the factors [1].
[
The total derivative of Y with respect to the factor , evaluated at the random state is:
( [
) ∑ ( [
) [ [
[ [ [
[
∑ ∑( [
) [ [
( [
) ( [
)
[ [ [
(4.4)
Where the derivatives represented by [ are derivatives of dependent random variables
[
(Section 2), and [ and [ are derivatives of independent random variables (Section 3).
Thus, the partial derivative of with respect to the factor [ , considering all other known
factors to be kept constant, evaluated at the random state is:
( [
) ∑ ( [
) ( [
)
(4.5)
The expected value of the partial derivative of is:
(( [
) ) ∑ (( [
) )
(4.6)
and its variance is:
(( [
) ) (∑ ( [
) )
(4.7)
If all predictor functions are independent of each other, then Eq. (4.7) becomes:
(( [
) ) ∑ (( [
) )
(4.8)
Now, in order to obtain the probability density function of [ it is necessary first to identify
the probability density functions of [ and [ . Once those functions are known, the
multivariate change of variable theorem [3] can be used to obtain:
[ [ [
( [ ) ∫ ∫ ∫ ∏( ( [ ) [ )
[ [
[ [ [
[ ∑ [
( )
[
(4.9)
The simplest case considered is the simple linear model given by:
(4.10)
Please notice that in a simple linear regression, the term represents the residuals of the
model, and is a standard normal random variable.
( ) ( )
(4.11)
Please notice that a partial derivative cannot be considered even though there are two random
variables ( and involved in the function. This is because the state of cannot be kept
constant as required by the partial derivative, since it is the result of all unknown factors
influencing . Thus, the expected value of the derivative is:
(( ) )
(4.12)
and its variance is:
(( ) )
(4.13)
The probability density function of the derivative is in this case (using Eq. 3.13):
( )
( )
√ √
( ) ( )
( )
√ √
√ ( ) √ ( )
(4.14)
√
which corresponds to a normal random variable with mean and standard deviation .
Let us now consider the following two-factor linear regression model without interactions:
[ [
(4.15)
[
The total derivative of with respect to is:
[
( [
) ( [
) ( [
)
(4.16)
Its expected value is:
(( [
) )
(4.17)
and its variance:
( [ )
(( [
) ) [
(4.18)
[
where [ is the variance of .
[ [
Assuming that behaves as normal random variable with mean and standard deviation
[
, and is also normal, then their derivatives are zero-mean normal random variables as
well as their sum, and therefore Eq. (4.16) can be expressed as:
( [
)
(4.17)
where represents a standard normal random variable and:
√ ( [ )
[
(4.18)
Thus, the probability density function of the total derivative can be determined by:
( [ )
[
( [ ) ( )
[ √
(4.19)
[
[
Now, the partial derivative of with respect to is (from Eq. 4.16 when ( [ ) ):
( [
) ( [
)
(4.20)
Although we are now dealing with partial derivatives, essentially the same result was obtained
when compared to the simple linear regression model. Thus, the partial derivative behaves as a
√
normal random variable with zero-mean and a standard deviation of [ .
( )
If, on the other hand, the interaction is considered, then the regression model becomes:
[ [ [ [
(4.21)
[
The total derivative of with respect to is now:
[ [
[ [ [
( [
) ( [
) ( )( [
) ( [
)
[
[ [
( )( [
) ( [
)
(4.22)
If the state is random, then the expected value of the total derivative of with respect to
[ is:
[
(( [
) )
(4.23)
and its variance:
[
( [ [ (( ) [ ) )
(( [
) ) [ [
(4.24)
[ [ [
If only the initial state of is known, that is but all other factors are random, then
[
the total derivative of with respect to becomes:
[
[ [
( [
) ( )( [
) ( [
)
(4.25)
with expected value:
[
(( [
) )
(4.26)
and variance:
[
(( ( ) ) [ )
(( [
) ) [ [
(4.27)
Now, if the initial state is completely known, , then the derivative is:
[
[ [
( [
) ( )( [
) ( [
)
(4.28)
(4.29)
and variance:
[
(( ( ) ) [ )
(( [
) ) [
(4.30)
[
which is a normal random variable when the noise and are normally distributed.
[
On the other hand, the partial derivative of with respect to is:
[
( [
) ( [
)
(4.31)
The expected value and variance for this partial derivative will be:
[
(( [
) )
(4.32)
(( [
) ) [ [
(4.33)
[ [
If the initial state of [ is known, , then the partial derivative and its expected value
and variance become:
[
( [
) ( [
)
(4.34)
[
(( [
) )
(4.35)
(( [
) ) [
(4.36)
When the initial state of [ is a normal random variable, the probability density function of
the partial derivative can be obtained from Eq. 4.8:
[
( [ )
[ √
( [ )
( )
[ [
( [ ) ∫ [ ( )
[ √
[ √ ( [ )
[ [ [
( ) ( [ )
[
[
( ) √
( [ ) [
∫ [
√ √
√ ( [ ) [
[ [
( )
[
( [ ( ) )
√ [
( [ )
√ [ √ [
(( [ ) ( ) ) (( [ ) ( ) )
√ √
√( √ [
√( √ [
[ ) ( ) [ ) ( )
(4.37)
That is, it is a normal random variable with the expected value and variance given by Eq. (4.32)
and (4.33).
(4.38)
where the rate of formation of component is given by the following second order Arrhenius-
type expression:
[ [
(4.39)
Let us assume that we want to analyze the behavior of the derivative of the reaction rate with
respect to the reaction temperature, when the temperature, kinetic parameters and the
concentrations of the reactants are normal random variables. Please notice that for this
example, the noise term containing the effect of unknown factors will not be directly
considered. However, they may have an effect on the variation of the factors considered.
[ [ [ [
( )
[ [
[ [ ( ) [ ( ) [ [ ( )
[ [ [ [
( ) ]
(4.40)
where the subscript 1 indicates the value of the random variables when , and the
subscript 2 indicates the value of the random variables when .
[ [
(( ) ) ( )
(4.41)
The expected value for the function of activation energy can be obtained using series
expansions of standard random variables [8], and using to represent a standard normal
random variable:
( ) ( ) (∑ ( ))
∑ (( ))
( ) ( ) ( )
∑ ∑
( )
(4.42)
(4.43)
The variance of the derivative is:
[ [
(( ) ) ( [ ( ) ) ( [ ( ) )
([ [ ( ) ) ( [ [ ( ) )
( [ [ )
( )
(( )( [ [ ) [ ( )( [ [ ) [
( )( [ [ )( [ [ )
( [ [ )( [ [ ) )
(( )( )( [ [ )( [ [ ) ( )
[ [ ( ) )
(4.44)
Using series expansion [8] for the expected value of the exponential functions of activation
energy, and simplifying terms the variance becomes:
(( ) ) ( )
( )( [ [ ) [ ( )( [ [ ) [ ( [ [ )( [ [ )
( )( [ [ )( [ [ )( ( ))
[ [
( )
]
(4.45)
Considering that the derivatives of the initial concentration with respect to temperature are
also normal, the derivative of the reaction rate with respect to temperature is also expected to
be approximately a normal distribution. The resulting full analytical expression for the
probability density function of the derivative is very large and it is not shown here; however, it
can be approximated by:
( (( ) ))
(( ) )
( )
√ (( ) )
(4.46)
As a particular example, Lin [9] reported the experimental kinetic parameters for the reaction
in the temperature range between and . They found the
following second order kinetic parameters:
(4.47)
(4.48)
(4.49)
(4.50)
For this example, initial concentrations are assumed to vary normally with the following
parameters:
[
(4.51)
[
(4.52)
[
(4.53)
[
(4.54)
The experimental rate of change of the reaction rate with respect to a change in reaction
temperature of (corresponding to the assumed resolution of the temperature sensor) is
a random variable with an expected value of (from Eq. 4.43):
(( ) ) ( )
(4.55)
(( ) ) √ (( ) )
(4.56)
The expected value and standard deviation of the derivative of the reaction rate with respect
to reaction temperature, for the whole range of temperatures considered is presented in
Figure 7. A minimum variance in the derivative is observed at about .
Figure 7. Expected value (blue solid line) and standard deviation (red dashed line) of the
derivative of the reaction rate with respect to reaction temperature for the experimental
reaction conditions reported by Lin [9], considering a change in temperature of in
the range from to .
Figure 8. Probability density function of the derivative of the reaction rate with respect to
reaction temperature for the experimental reaction conditions reported by Lin [9], considering
a change in temperature of at a reaction temperature of .
5. Conclusion
The derivatives of functions of random variables are also random variables, characterized by an
expected value, a variance and a probability density function. The probability density function
of the derivative, for most cases, can be determined using the change of variable theorem [3].
For functions of a single random variable , the probability density function can be
determined by:
∑ ( )| |
(5.1)
where denotes the -th solution of the inverse function of , and is the total number
of different possible solutions of .
If is independent of , then the derivative will be a random variable with zero-mean, and
variance given by:
(5.2)
The corresponding probability density function is given by:
∫ ( )
(5.3)
The probability density function obtained is symmetrical around zero. If the independent
random variable is normal, its derivative (and higher order derivatives) will be normal.
(5.4)
The expected value, variance and probability density function of the partial derivative
( [ ) are given by:
(( [
) ) ∑ (( [
) )
(5.5)
(( [
) ) (∑ ( [
) )
(5.6)
[ [ [
( [ ) ∫ ∫ ∫ ∏( ( [ ) [ )
[ [
[ [ [
[ ∑ [
( )
[
(5.7)
If the probability density functions and are normal, then can be approximated
[ [ [
Acknowledgments
The author gratefully acknowledges valuable scientific discussions on this topic with Prof. Dr.
Silvia Ochoa (Universidad de Antioquia) and Prof. Jaime Aguirre (Universidad Nacional de
Colombia).
This research did not receive any specific grant from funding agencies in the public,
commercial, or not-for-profit sectors.
References
[1] Hernandez, H. & Aguirre, J. (2017). Calculus of Random Finite Differences and Differentials.
ForsChem Research Reports 2017-15. doi: 10.13140/RG.2.2.31562.67529.
[2] Hernandez, H. (2016). Variance algebra applied to dynamical systems, ForsChem Research
Reports, 2016-2, doi: 10.13140/RG.2.2.36507.26403.
[7] Hernandez, H. (2018). Statistical Modeling and Analysis of Experiments without ANOVA.
ForsChem Research Reports 2018-05. doi: 10.13140/RG.2.2.21499.00803.
[8] Hernandez, H. (2016). Modelling the effect of fluctuation in nonlinear systems using
variance algebra - Application to light scattering of ideal gases, ForsChem Research Reports,
2016-1, doi: 10.13140/RG.2.2.36501.52969.
[9] Lin, C. L. (1982). Temperature dependence of the rate constant for the reaction OH+ H2S.
International Journal of Chemical Kinetics, 14(5), 593-598.