Vector Calculus - Understanding The Gradient - BetterExplained

1/2/2015
VectorCalculus:UnderstandingtheGradient|BetterExplained
Search...
Vector Calculus: Understanding the Gradient

The gradient is a fancy word for derivative, or the rate of change of a function. Its a vector (a direction to
move) that
Points in the direction of greatest increase of a function (intuition on why)
Is zero at a local maximum or local minimum (because there is no single direction of increase)
The term gradient (grad) typically refers to the derivative of vector functions, or functions of more than
one variable. Yes, you can say a line has a gradient (its slope), but using the term gradient for singlevariable functions is unnecessarily confusing. Keep it simple.
Gradient can refer to gradual changes of color, but well stick to the math definition if thats ok with you.
Youll see the meanings are related.
Properties of the Gradient

Now that we know the gradient is the derivative of a multi-variable function, lets derive some properties.
The regular, plain-old derivative gives us the rate of change of a single variable, usually x. For example,
dF/dx tells us how much the function F changes for a change in x. But if a function takes multiple variables,
such as x and y, it will have multiple derivatives: the value of the function will change when we wiggle x
(dF/dx) and when we wiggle y (dF/dy).
We can represent these multiple rates of change in a vector, with one component for each derivative. Thus,
a function that takes 3 variables will have a gradient with 3 components:
F(x) has one variable and a single derivative: dF/dx
F(x,y,z) has three variables and three derivatives: (dF/dx, dF/dy, dF/dz)
The gradient of a multi-variable function has a component for each direction.

And just like the regular derivative, the gradient points in the direction of greatest increase. However, now
that we have multiple directions to consider (x, y and z), the direction of greatest increase is no longer
simply forward or backward along the x-axis, like it is with functions of a single variable.
http://betterexplained.com/articles/vectorcalculusunderstandingthegradient/
1/30
1/2/2015
If we have two variables, then our 2-component gradient can specify any direction on a plane. Likewise,
with 3 variables, the gradient can specify and direction in 3D space to move to increase our function.
A Twisted Example
Im a big fan of examples to help solidify an explanation. Suppose we have a magical oven, with coordinates
written on it and a special display screen:
We can type any 3 coordinates (like 3,5,2) and the display shows us the gradient of the temperature at
that point.
The microwave also comes with a convenient clock. Unfortunately, the clock comes at a price the
temperature inside the microwave varies drastically from location to location. But this was well worth it: we
really wanted that clock.
With me so far? We type in any coordinate, and the microwave spits out the gradient at that location.
Be careful not to confuse the coordinates and the gradient. The coordinates are the current location,
measured on the x-y-z axis. The gradient is a direction to move from our current location, such as move
up, down, left or right.
Now suppose we are in need of psychiatric help and put the Pillsbury Dough Boy inside the oven because we
think he would taste good. Hes made of cookie dough, right? We place him in a random location inside the
oven, and our goal is to cook him as fast as possible. The gradient can help!
The gradient at any location points in the direction of greatest increase of a function. In this case, our
2/30
1/2/2015
function measures temperature. So, the gradient tells us which direction to move the doughboy to get him
to a location with a higher temperature, to cook him even faster. Remember that the gradient does not give
us the coordinates of where to go; it gives us the direction to move to increase our temperature.
Thus, we would start at a random point like (3,5,2) and check the gradient. In this case, the gradient there is
(3,4,5). Now, we wouldnt actually move an entire 3 units to the right, 4 units back, and 5 units up. The
gradient is just a direction, so wed follow this trajectory for a tiny bit, and then check the gradient again.
We get to a new point, pretty close to our original, which has its own gradient. This new gradient is the new
best direction to follow. Wed keep repeating this process: move a bit in the gradient direction, check the
gradient, and move a bit in the new gradient direction. Every time we nudged along and follow the gradient,
wed get to a warmer and warmer location.
Eventually, wed get to the hottest part of the oven and thats where wed stay, about to enjoy our fresh
cookies.
Dont eat that cookie!

But before you eat those cookies, lets make some observations about the gradient. Thats more fun, right?
First, when we reach the hottest point in the oven, what is the gradient there?
Zero. Nada. Zilch. Why? Well, once you are at the maximum location, there is no direction of greatest
increase. Any direction you follow will lead to a decrease in temperature. Its like being at the top of a
mountain: any direction you move is downhill. A zero gradient tells you to stay put you are at the max of
the function, and cant do better.
But what if there are two nearby maximums, like two mountains next to each other? You could be at the top
of one mountain, but have a bigger peak next to you. In order to get to the highest point, you have to go
downhill first.
Ah, now we are venturing into the not-so-pretty underbelly of the gradient. Finding the maximum in
regular (single variable) functions means we find all the places where the derivative is zero: there is no
direction of greatest increase. If you recall, the regular derivative will point to local minimums and
maximums, and the absolute max/min must be tested from these candidate locations.
The same principle applies to the gradient, a generalization of the derivative. You must find multiple
locations where the gradient is zero youll have to test these points to see which one is the global
maximum. Again, the top of each hill has a zero gradient you need to compare the height at each to see
which one is higher. Now that we have cleared that up, go enjoy your cookie.
Mathematics
3/30
1/2/2015
We know the definition of the gradient: a derivative for each variable of a function. The gradient symbol is
usually an upside-down delta, and called del (this makes a bit of sense delta indicates change in one
variable, and the gradient is the change in for all variables). Taking our group of 3 derivatives above
Notice how the x-component of the gradient is the partial derivative with respect to x (similar for y and z).
For a one variable function, there is no y-component at all, so the gradient reduces to the derivative.
Also, notice how the gradient can itself be a function!
If we want to find the direction to move to increase our function the fastest, we plug in our current
coordinates (such as 3,4,5) into the equation and get:
So, this new vector (1, 8, 75) would be the direction wed move in to increase the value of our function. In
this case, our x-component doesnt add much to the value of the function: the partial derivative is always 1.
Obvious applications of the gradient are finding the max/min of multivariable functions. Another less
obvious but related application is finding the maximum of a constrained function: a function whose x and y
values have to lie in a certain domain, i.e. find the maximum of all points constrained to lie along a circle.
Solving this calls for my boy Lagrange, but all in due time, all in due time: enjoy the gradient for now.
The key insight is to recognize the gradient as the generalization of the derivative. The gradient points to
the maximum of the function; follow the gradient, and you will reach the local maximum.
Questions
Why is the gradient perpendicular to lines of equal potential?
Lines of equal potential (equipotential) are the points with the same energy (or value for F(x,y,z)). In the
simplest case, a circle represents all items the same distance from the center.
The gradient represents the direction of greatest change. If it had any component along the line of
equipotential, then that energy would be wasted (as its moving closer to a point at the same energy). When
the gradient is perpendicular to the equipotential points, it is moving as far from them as possible (this
4/30
1/2/2015
article explains why the gradient is the direction of greatest increase its the direction that maximizes
the varying tradeoffs inside a circle).
Other Posts In This Series

1. Vector Calculus: Understanding Flux
2. Vector Calculus: Understanding Divergence
3. Vector Calculus: Understanding Circulation and Curl
4. Vector Calculus: Understanding the Gradient
5. Understanding Pythagorean Distance and the Gradient
6. Vector Calculus: Understanding the Dot Product
Vector Calculus
Printable version
Better Explained is dedicated to helping you grasp concepts, and serves
over 250k readers each month.

Enjoy this article? Try the site guide or join the newsletter:
you@youremail.com
Get Free Lessons + Bonus PDF
153 Comments
1. me
i like it well explained.
2. Jane
Super!!!
3. Chris
You are the man! Nice work!
Kalid
5/30
1/2/2015
4. Kalid
Thanks, glad it was helpful for you.
5. gaurav
i was always looking for conceptual and practical examples and yes i finally got.
6. Harry
Awsome!
7. Palo
well you made a good explanation, that even a not-so-smart guy gets it, but i think you missed the
obvious -> WHY does gradient show the direction of the greatest increase.
I think that the principle of the gradient is quite easy, but understanding why does it work the way it
does is a bit tricky and you should have focued on it more.
It would be interesting if you would somehow add it to this good article. Inspiration
http://mathforum.org/library/drmath/view/68326.html
good luck !
8. Kalid
Hi Palo, thats a great point! Ive been feeling a bit guilty, if you can imagine it, because Ive lacked that
explanation
Im probably going to do a separate article on the reason *why* the gradient points in the direction of
greatest increase I have another explanation that it works well with. Thanks for the link and feedback!
9. John Gabriel
Your introduction is not quite correct:
You claim: Points in the direction of greatest increase of a function.

6/30
1/2/2015
Why? It can also point in the direction of greatest decrease of a function.

A gradient is one or more directional derivatives. These derivatives are considered in a particular
direction. In the case of single variable calculus, we generally talk about a directional derivative when
we consider multiples of the x unit vector, i.e. k*(1,0). To consider the y unit vector, we deal with the
partial derivatives with respect to y in a given direction. In three dimensions, the 3 partial derivatives
form what we now call a gradient.
So in fact it is incorrect to call this a slope or anything else except to say that it describes the partial
derivatives of a point in the direction of a given vector in space.
Does this make sense? Please visit my blog for some more interesting reading.
http://mathphile.blogspot.com/
10. Kalid
Hi John, thanks for writing. Youre right, the formal definition of a gradient is a set of directional
derivatives.
But when thinking about the intuitive meaning, I think its ok to consider the gradient as a vector that
points in the direction of greatest increase (i.e. if you follow that direction your function will tend
towards a local maximum).
Unless Im mistaken, the gradient vector always points in the direction of greatest increase (greatest
decrease would be in the opposite direction).
11. John Gabriel

What I was saying is that it points either one way or the other, it is not restricted to the direction of
greatest increase. As a simple example, consider what happens when you differentiate a parabola: You
set the derivative equal to 0 and then you determine that it has either a maximum or a minimum at its
turning point. It is not always a maximum just as it is not always a minimum. Think I have explained
this correctly now.
12. sqib
good john you have done a great job.
7/30
1/2/2015
13. Kalid
Hi John, thanks for the clarification. Id still politely disagree and say that in general, the gradient points
in the direction of greatest increase :).
In the case of 2 dimensions, the gradient/slope only gives a forward or backward direction. A positive
slope means travel forward and a negative slope means travel backwards.
Consider f(x) = x^2, a regular parobola. The gradient is zero at the minimum (x=0), and there is no
*single* direction to go. At x = -1, the slope is negative, which means travel backwards (to x = -2) to
increase your value. Similarly, at x = 1, you travel forward (to x = 2) to increase your value.
But, as you mention, strange things can happen when the derivative = 0. It can mean you are at a local
maximum (no way to improve), or at a local minimum (no single direction to improve your position
forward or back will help). I consider the corner case of zero an exception to the general rule / intuition
that the gradient is the direction to follow if you want to improve your function.
14. Vidhya
Wonderful explanation!
15. Kalid
Thanks Vidhya, glad you liked it.
16. bihazo
hi john keep it up you done a great job
17. Travis
Thanks a bunch! I didnt think it could be this simple to find the maximum increase at a point, so I
thought Id look it up. Thanks to your great explaination, it turn out it was as easy as it seemed it should
be. Great job! Thanks!
Travis
18.
Kalid
8/30
1/2/2015
Awesome, glad it worked for you
19. caitlyn
thanks!!!!
20. Kalid
Hi Caitlyn, youre welcome.
21. Derek
Thanks! The sadistic microwave example helped a lot.
22. Kalid
Awesome, glad it was useful :).
23. John Gabriel

Hello Kalid,
Did not read your reply for some
time. Am sorry you do not agree.
Let me give you an example:
Suppose we are dealing with pressure
and height in a certain cubic
area. Suppose that the middle of the
cube height is 0 meters. Also suppose
that we have a whirlpool generated in the
cube such that the pressure rate increases
as we go below the middle of the cube.
Anything below is negative height and anything above
is positive height. Now, as one rises
higher in the cube, the pressure decreases.
If we find the gradient, then according to
your definition (and many others), then
the gradient vector for the rate of greatest
increase will point below the middle of the
9/30
1/2/2015
increase will point below the middle of the

cube, not above. But above the middle we
find the greatest decrease in rate of pressure.
In this example, greatest increase points
downwards and greatest decrease upwards.
It would probably be better to define
gradient as a vector that points in a
direction of greatest increase or decrease.
Its additive inverse will point in the
diretion of greatest decrease or increase
respectively. For most physical phenomena,
your definition would generally be true.
But what happens when you have an anomaly?
Make sense?
24. John Gabriel

I do not believe I have the best answer to this question but like yourself, I am a believer in trying to find
the best possible explanation. Once again, I like your website. Keep up the good work Kalid!
25. John Gabriel

Okay, I think I have the best answer. If f is a real-valued function, then del(f) or gradient of f points to
the greatest increase, whereas -del(f) points t0 the greatest decrease.
For once planet math has some decent information on this since I last checked:
http://planetmath.org/encyclopedia/Gradient.html
I do not endorse everything Planet Math publishes but this particular information appears to be correct.
In any event, it clears up the previous confusion I think.
26. Kalid
Hi John, thanks for the comment! Yes, thats an important distinction to make: the positive gradient is
the greatest increase, and the negative gradient is the greatest decrease. Thanks for helping clarify :).
27. Jared
10/30
1/2/2015
Thank you!
28. Bigmouth
This actually makes sense to me. Thanks!
29. Kalid
@Jared, Bigmouth: Cool, glad it was helpful!
30. Anonymous
did not grasp the idea
31. Kalid
Be more specific. The gradient is the direction to move that gives you the biggest increase.
32. Shaheen
It helps me a lot. But I have some doubt still now.Is it the same concept for gradient of each vertex in a
triangle mesh?
Thanks so much.
33. JohnnyT
Kalid
Thanks for the great explanations! I thought I was math-retarded for some time; however your writings
actually make sense to me!
Take care!
Johnny T
11/30
1/2/2015
34. Kalid
@Shaheen: Thanks, glad you enjoyed it. Im not sure I understand the question: in a triangle mesh, you
could measure the gradient at each vertex to find the best direction to move. Again, not sure if this is
your question.
@Johnny T: Thank you for the comment! Yes, when a subject seems difficult (as vector calculus was for
me) sometimes its just because the explanation wasnt clicking properly. Thanks for dropping by.
35. wali khan

well done,excellent explaination with solid examples
36. Kalid
Thanks Wali, glad you enjoyed it.
37. j.sathish kumar

thanks
but i have some doubts.how the differentaion gives the maximum space rate of change. as per my
understandings differentiation only is difference between two point in the region say p1 and p2.can u
clarify
38. leon
Thanks a lot for explaining the concept.
39. sophie
i was having so much trouble understanding this and now its all clear thank you so much!
40. Kalid
@lon, sophie: Thanks, glad you enjoyed it!
Ryan Johnson
12/30
1/2/2015
41. Ryan Johnson

Jesus. This was a lot better explained than in my text book and by my professor. I thought we were using
the gradient as the normal vector but I really doubted that it could be that.
42. Kalid
@Ryan: Thanks! I struggled with this concept for a while also.
43. Ranjeet Kumar

thanks ! this explanation made me clear how to find the direction of smallest change.It is just the 90
degree rotation of gradiant(the direction of largest change).
44. Shakeel Ahmed

Thanks very much for your effort
45. Bill
Um in your microwave example, arent you pushing the doughboy out the back of the microwave?
(Just wanted to understand the concept). I love these essays, btw, keep them coming!
46. Hehehe
I loved the microwave analogy.also thanks for clarifying the upsidedown delta now everything makes
more sense
47. RAHUL
stil im confused between scalar field and vector field.
48. aradhita chattopadhyay

how can such a mathematical expression denote the max change? pls i didnt understand the relation of
this with mathematics. pls reply sir.
13/30
1/2/2015
49. nat2_bam2
thank you soo much!!
its a big help for our project
Can we have your number?hehe
50. Kalid
@Rahul: A scalar field returns a single value (x), but a vector field returns multiple values (x,y,z).
Usually the multiple values (x,y,z) are taken as a direction to follow.
@aradhita: Hi, thats a question I need to get into in a later post.
@nat2_bam2: Thanks!
51. Migs
Hi kalid! i read your explanation. oh this is very helpful! by the way can you give an example on how to
apply this on a situation of the classic mountain and mountain climber problem? hope you will reply.
thanks again your explanations were clear
52. Kalid
@Migs: Great question. The classic mountain climber problem is when the vector field gives the
height of the mountain (z) at a certain position (x,y), so z = f(x,y).
The gradient at any position x,y will give you the direction of the _greatest increase_ in z. That is, the
gradient will point in the most uphill. Following the gradient will give you the shortest path the the
top of the mountain (technically, the top of the nearest local maximum). How this helps!
53. vignesh
beautifulwell said
54. akansha
thanks a lot for the wonderful explanation!!!
14/30
1/2/2015
thanks a lot for the wonderful explanation!!!
55. Kalid
@akansha: Youre welcome!
56. anonymous
Very nice! Keep up. Thanks a lot
57. Florencia
Very nice article!!
Hope to see how to find the maximum of a constrained function soon!!
Thanks a lot!!
58. Kalid
@Florencia: Glad you liked it! Thanks for the suggestion.
59. ab
Very good explanation by the way. So if you are on a landscape given by z=cosy-cosx and u want to get
from (0,0,0) to (4pi,0,0) by moving in the direction of the gradient in the positive x-direction how would
u explain that? What would that path look like?
60. P-F
Thanks for the great explanation. Another topic that would be very interesting for you to cover is the
Jacobian, which causes pain for many, many students (including myself).
61. Kalid
@P-F: Thanks for the note I think the Jacobian, and linear algebra in general, would be great to cover.
Ive forgotten a lot of it and am looking to relearning :).
Mark Soric
15/30
1/2/2015
62. Mark Soric

Just wondering something. In that case of f(x,y) = X^2 + y^2, a paraboloid how can the gradient by
perpendicular to the tangent plane at all point and only have components in x and y
gradF(X,Y) = 2x + 2y
How can it point in any other direction other than parallel to the xy plane?
Im lost here.
63. prabu
thank you kalil. wonderful explanation.
64. Kalid
@prabu: Glad it helped!
65. Ashraful
It was a great explanation! But I have a specific problem with gradients. Is there any functions that cant
be expressed as gradient of any parameter? What could be the properties of that function?
66. Ashraful
May I could be more specific about my previous problem. If a function is constant in all direction, is it
possible to express the function as gradient?
67. Kalid
Im not sure if I understand the question the gradient of a constant function would be a 0 vector
[perhaps technically (0,0)], that is, there is no direction of greatest increase. If it helps, think of the
gradient in terms of a derivative (the derivative of a constant function is 0).
68. Kinar
Math professional!
16/30
1/2/2015
69. Anonymous
Thank you for getting to the heart of why del is required and how to intuitively understand it. Its the
first time I understand it so well despite reading so much about it before!
70. Anonymous
damn! i got it now
71. Anonymous
math is so beautiful
72. bob clear

WOW! great explanation. thanks dude.:D
73. Kalid
@bob: Thanks!
74. Kalid
@Anonymous: Agreed :).
75. Jose
Great explanation helped me explain my brother! Nice job! Gonna bookmark it for further needs I might
have with it.
76. js
great explanation and example
Kalid
17/30
1/2/2015
77. Kalid
@js: Thanks!
78. shreedhar
hey, explained really well. But still you didnt provide any sign of why the gradient would always point in
the direction of maximum increase
79. Nick Pellatz

I dont usually comment on blogs, but this is a great explanation. Way better than my text book.
A+++++++++
80. Kalid
@shreedhar: Thanks Id like to cover that in a follow-up article. I need to get a nice, intuitive
explanation for it first ;).
@Nick: Thanks, glad it helped.
81. Al Paquette
Man! I just love this kind of explanation. Its so clear and concise, and it shows me that the author really
understands the concept himself.
All mathematics should be taught this way. Go from the specific to the general (abstract). Not the other
way around, which is the path usually followed by the type who wants to show off his prowess with math
symbols and equations.
82. Anonymous
nice explanation
83. Anonymous
dont eat that cookie!
18/30
1/2/2015
84. Kalid
@Al: Thank you! I think one of the big problems in math teaching (especially) is just trying to get things
explained without the professors prowess getting in the way, as you say.
@Anonymous: How could you eat cookies when theres gradients to be studied?
85. Pandia
Nice work!! Thanks man:)
86. Kalid
@Pandia: No prob!
87. Gaton
Thanks!
88. Marwa
Thanks alot,I loved your way explaining this, very helpful indeed.
Keep it up.
89. Kalid
@Marwa: Thanks, glad it helped!
90. Vitor P.F.

Great !! Congrats
91. Alex
Awesome! I was cracking my head trying to figure out HW, only to realize how basic it was after reading
through ur page. Thanks!
19/30
1/2/2015
92. dianne
goodluck with my exam tom. ^^
93. Anonymous
It wasnt a bad explanation but I wish you had explained why the gradient is the perpendicular vector
of the function its derivatives were derived from. This still bothers me a little.
Also, if we have a function with three variables, shouldnt the independent variable be considered? By
considered I mean, if I have a function F(x, y z), then I am saying that w = F(x, y, z), and this function
can not be graphed since it has 4 dimentions. A normal F(x, y) can be graphed since you considered the
Z, X, and Y of the graph.
From the book I read, I interpreted that the original function has a constant value for w, hence
producing a graph with a new function F2(x, y). However I still didnt see the math that proves that the
gradient of the function F(x, y, z) is actually the vector that is perpendicular to the surface of the graph
from which its derivatives were derived from. If you could prove this, it would be really helpful.
94. mohamed
thx man very much
I understood it totally from u
my regards
95. Burton
Thank you very much! This made perfect sense and it really helped me out.
96. Kalid
@Burton: Thanks!
97. panchito
Cool! Thanks
20/30
1/2/2015
98. Donglin
good explain, it solved my problem
99. Anonymous
kalid , are u professor
100. shrikant
such wonderful explanation..wow
101. Kalid
@shrikant: Thanks!
102. jayakumar.g
so very easy method
103. Mrigeh
I love you!
104. Kalid
@jayakumar.g: Glad it helped
@Mrigeh:
105. hyaa
i want to ask, once knowing the maximum rate of change of temperature in yours microwave example,
how we can attain that particular place without moving our coordinates positions as mentioned by
microwave for example when we choose coordinates (3,5,2) we obtain gradient as (3,4,5). now from
where we get the information that which coordinates should be selected next time that gives us
maximum gradient? should we choose (3,4,5) coordinates?
21/30
1/2/2015
maximum gradient? should we choose (3,4,5) coordinates?
106. Kalid
@hyaa: Im sorry, I dont think I understand the question. The gradient gives you the direction (not
coordinates) of the greatest increase in your current value. You have to follow the gradient for a bit, get
to a new point, get the gradient there, follow it for a bit and so on to maximize your value.
Think of the gradient as a compass which points towards your greatest increase. A compass doesnt give
you the coordinates of North, but tells you how to get there from your current position. Hope that helps.
107. Zita
Hi, I still have a question. If there is a function h(x,y)to denote the height of a mountain at position(x,y).
Can I use the knowledge of gradient to locate the top of the mountain and how?
108. kalid
@Zita: Yep you start at any point, and keep following the gradient of h to find the top.
109. max
great, really good thank you,it would be comprehensive if you explain that why the gradient is the
perpendicular vector of the function its derivatives were derived from
110. kalid
@max: Great question. Going to add it as a Q & A at the end of the article.
111. Patel Ankit

Excellent explanation, I think if you provide your ebook for free of cost it would really be helpful for the
poorer students to strengthen thier grass-roots.
GOOD JOB, KEEP IT UP.
112. Chico
Consider the directional derivative, f_u.
22/30
1/2/2015
Consider the directional derivative, f_u.
f_u = f_x u_1 + f_y u_2 (it takes some effort to see this definition of f_u)
=grad(f) dot u (u is a unit vector)
=|grad(f)| cos@ (@ is the angle between grad(f) and u)
Thus, it is clear that the directional derivative, f_u, is maxed when cos@=1.
It follows that @=0 and the directional derivative, f_u, is attained when u is in the direction of the
gradient. Therefore, the gradient does indeed give the direction of greatest increase.
Note that f_u is minimized when cos@=-1. Thus, @=pi, and u is in the opposite direction of the
gradient. QED
ps
I am a nerdy math professor who likes demonstrating mathematical prowess. Thanks for the microwave
intuition builder. My students are going to like that.
113. Deniz
I already knew this but you gave me a better intuition of it and I like your style of writing! Thank you!
114. kalid
@Chico: Awesome, thanks for sharing! I like that a lot lining up with the gradient (out of all possible
directional derivatives) will give you the best return (cosine = 1). That clicks for me.
Glad you enjoyed the microwave intuition, I love searching for little analogies.
115. kalid
@Deniz: Thanks! And youre welcome :).
116. beant singh

ah highly informative and excellently defined
117. kalid
@beant: Great, glad it helped!
madhu
23/30
1/2/2015
118. madhu
that cookie tasted awsome
119. madhu
could u explain why does gradient is zero for local minimum?
120. kalid
Madhu: Those cookies are delicious :). The gradient behaves like the first derivative (rate of change). In
regular calculus, d/dx = 0 means your function is not changing [therefore you are at a max or min].
Similarly, when the gradient is zero, it means your function is not changing when you move.
121. Anonymous
wow gr8
122. Yogesh
Hey, could you explain WHY gradient points in the direction of maximum increase?
I mean given that this article is mindblowing but I still cant get Why maximum increase..
Also, could you explain a vector field? In case of scalar field, what I imagine is as follows: Consider the
3D space, and for each point my scalar function returns a value.. And with a intensity proportional to
that value, a black point appears(Greyscale).. I cant picture anything about a vector field though.. Please
consider..
And again, thanks for all the Vector calculus explanations..
123. kalid
Hi Yogesh, thanks for the note.
This article explains why the gradients points in the direction of greatest increase:
http://betterexplained.com/articles/vector-calculus-understanding-the-gradient
The essence is you have a circle of possible directions, the individual derivatives (df/dx and df/dy) give
you the tradeoff as you change directions, so find the direction that makes the best use of that tradeoff.
A vector field is tricky. Imagine your same 3d space, but instead of a point (a single value) imagine that
there is wind blowing through it. Each position in
your space could feel a different push (strength and
24/30
1/2/2015
there is wind blowing through it. Each position in your space could feel a different push (strength and
angle) from the wind. For example, in a cyclone, the push might be in a circle (each individual point is
pushed in a different instantaneous direction), with a dead spot in the middle. With a steady wind, every
point might feel the same push.
124. someone
You are a gift to mankind! Well to me you are.
125. Jose
People confused by gradient pointing towards maximally increasing direction, or perpendicularity of
gradient to equipotential.
When you have a function, you know what I mean by function, something that looks like a hill, whether
its a 2d hill you drew on the paper or a 3d hill you made out of clay on your table, stick your finger at a
single point on the hill. Now ask what is the change height of the hill as I move across the hill a little
bit in a flat direction? In the 2d case, you go left or right, the only flat direction. You trace the line
one inch to the right (right = positive), following it, and you find the line goes up (up = positive) one
inch, so you say the slope is one inch (up) per inch (to the right) near that point. The per means divide.
Positive / positive = positive. Going back to the left (negative) you say you went down (negative) one
inch. Neg/Neg = Positive. The gradient for this 2D example, since its defined a vector pointing along x,
can only point along the x axis, but which way? Left or right? The gradient also has a magnitude, which
can be very positive, a little positive, a little negative, or very negative. Its magnitude is whatever you
got for the slope. When the slope was positive, the gradient will point along x, for sure, but along the
positive direction. When the slope was negative, the gradient still points along x, but towards the
negative direction (left). So the gradient points along x towards the direction of increasing height,
whichever way it is. If the increasing height was to the left of your finger, you would find the slope was
negative there after defining left to be negative and up to be positive, and the gradient would point
negative along x, saying look at me, Im pointing the direction you travel along x to see the hill rise.
In the 3d case, the clay on your table, you stick your finger and soon realize you get different slopes
depending on if you move your finger forwards, backwards, left, right, or diagonal. Lets say you move
your finger to the left and it goes up. Then the portion of the gradient pointing to the left is the slope
your finger measured, since your finger went up moving that direction a component of the gradient
points somewhat in that direction. Then you go back to where you started from. You go forward, and
find your finger went up even more than it did going left. Lets say your finger went up two inches going
an inch forward, and it went up one inch going an inch to the left. We know a portion of the gradient
points forwards (cause your finger went up, not down), and about half that portion (you went up an inch,
not two inches like going forwards) points to the left. So the gradient points very forward, and a little
left (2 inch up per inch forward, pointing forward, 1 inch up per inch left, pointing left). You will notice
this is obviously uphill. It turns out it is EXACTLY uphill. If you go 2 forward for every 1 left you move
near that point, you will gain the most height possible. When you gain the most height possible, you are
moving exactly perpendicular to the direction you
would not gain height if you were moving, aka the
25/30
1/2/2015
moving exactly perpendicular to the direction you would not gain height if you were moving, aka the
equipotential path, or path of same height.
126. Baljeet Kaur

its awsomeit clearly explains the actual physical significance of a gradient..thank you..:)
127. kalid
Hi Baljeet, glad it helped!
128. jayakrishnan.k.j
BRILLIANT EXPLANATION! Thanks a lot
129. manju
thks for complete explanation
130. Kevin @ http://kldavenport.com
I stumbled across your site looking for one specific aspect of gradients and ended up reading the whole
post. You did a great job of distilling these concepts Kalid.
131. Kalid
Thanks Kevin, I appreciate it!
132. Rick
I went over to Wikipedia and read an article on a similar topic. It was so much more difficult to
understand, and Wikipedia is easier than most math texts. It makes me so angry that most math books
seem to go out of their way to make mathematics unnecessarily difficult. Maybe, with more people
beginning to write internet articles like this, the math obfuscators wont be able to get away with it
much longer. I look forward to the day when students realize that math can be the easiest class in
school.
26/30
1/2/2015
Thanks and keep up the good work Kalid!
133. Kalid
Thanks Rick! I can relate to the feeling of frustration, its what drove me to write (after having an aha
momne, it quickly turned to: Why couldnt they explain it like that in the first place?).
I do think math has the potential to become the easiest subject. Its objectivity, which could be seen as
offputting, is a great indicator of when something has truly clicked. As a result, we can quickly
determine whether an analogy is helping solve the problem before us.
Appreciate the support :).
134. Marco P
Lovely!
135. Khasud
Jazak Allah khair
136. Qammar Abbas

An easy and interesting approach. Hats off!
137. ATHUL P ANAND

awesme dude!!gud work!!!well explainedws really helpful!
138. lashiwe
Thank u so much,your explanations really helped- must confess am not all that good in mathematics I
could really use some more help.but thank you
139.
Kalid
27/30
1/2/2015
Another explanation that I posted on reddit:

I imagine a vector field like a grid. Off in the distance is a billboard showing the amount of money well
get for being at a certain position on the grid (value of the function).
From where were standing, we can take a step in any direction. The work is the same (i.e., I moved 1
unit) but the payoff (net amount of cash we gain) can differ drastically depending on if we go North,
West, South, NorthEast, etc.
The gradient is the payoff in the sense it points to the direction of greatest reward (if its zero, it
means you are already at the max reward, i.e. any step you take will diminish your earnings).
A surface like z = 3 is basically saying show me all the positions in this grid where the reward is $3.
On the grid a path is drawn, highlighting all the positions of this equal payoff. If you started following
this path, your payoff would never change.
But what would the gradient be? The gradient is pointing in the direction of greatest increase, so
should have nothing in common with the path of 0 increase. In other words, the projection of the
gradient onto the surface should be 0, i.e. its normal. The gradient has zero inclination for you to go
anywhere near the path of zero gains.
Ok, fine, thats what the gradient should do. How do we show it actually maximizes the payoff?
Heres how I figure it: on a circle (showing all possible paths), we can basically make any tradeoff of x
and y that we want. At 45-degrees we can trade them 1-for-1, at higher values we can get 2 units xdistance for 1 unit of y distance, or 10 to 1, or a million to 1 (at angles close to 0 or 90).
The direction of the gradient is calculated to maximize the tradeoff based on dz/dx and dz/dy, i.e. it
figures out how much reward we get for moving in each direction and allocates effort appropriately.
If we get $20 for moving the x direction and $10 for moving in the y direction, then our direction should
favor x, but only at a 1:2 tradeoff. I.e., if we can trade 1 x for 3 ys then we should keep trading (adjusting
the angle) until we get 1x for 2ys.
In other words, you can prove that the gradient direction is the direction which maximizes z assuming
you are moving 1 infinitesimal unit and are getting rewarded by dz/dx and dz/dy. And by definition, this
profit-maximizing direction would not waste any energy along the profit-maintaining path that must
have both dz/dx and dz/dy of 0 (the equal-valued path must not change the amount of z).
140. Rob
Really great explanation! The only thing is that in your definition of the del the partial derivatives
should have a lower-case delta ( instead of d
28/30
1/2/2015
should have a lower-case delta ( instead of d
141. Ajinkya
Really helpful
I would be really glad if you could tell me the derivation for the formula for gradient of a scalar in terms
of the nabla/del operator
Or if you could tell a link where I can find it
142. pari
Hi kalid, it is a great explanation at lest for people like me.
I think one should get this overview before getting into the actual concept.
thanks a lot!
143. Naveen
Thanks a bunch !
144. Rajmohan
very good article
145. Aymun
MashaAllah you posted this in 2007and to this date people are getting benefit from it. Loved it. Thank
you
146. Anonymous
what is greens theorem? can anyone explain.
147. sanu
Before reading this, I wasted 2 days for this gradient!!
but the world of mathematics is very very very interesting when people like you teaching us.
thanks a lot.
29/30
1/2/2015
148. Osama
Awesome work But if you can help me with this problem,? My professor assigned me physical
significance of gradient that would be my first assignment confused what to submitt? ??
(application/physical significance)
149. Amir sultan

Good explanation
150. Taps patel

Thank you so much
151. J Scrib
Thanks a lot. this really helped me understand it better!
152. Manikanta
A gradient is Vector differentiation operator applied on a scalar function. Not strictly the derivative of
vector functions as you said in opening paragraph, both are different. However, gradient can be treated
as a derivative of a special vector for which all the vector components have same function, whose
magnitude is the scalar function and directed in the direction of f_x=f_y=f_z from notation f(vector) =
(f_x,f_y,f_z).
153. Manikanta
Iam adding this Just to clarify the things,
1. Derivative of a vector function is called divergence.
reference:http://ocw.mit.edu/ans7870/18/18.013a/textbook/HTML/chapter09/section01.html,
2. Gradient of a vector function is called Jacobian.
reference:http://en.wikipedia.org/wiki/Gradient#Gradient_of_a_vector
30/30

Vector Calculus - Understanding The Gradient - BetterExplained

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Vector Calculus - Understanding The Gradient - BetterExplained

Uploaded by

Copyright:

Available Formats

1/2/2015

Vector Calculus: Understanding the Gradient

Properties of the Gradient

The gradient of a multi-variable function has a component for each direction.

Dont eat that cookie!

Other Posts In This Series

Better Explained is dedicated to helping you grasp concepts, and serves

over 250k readers each month.

Get Free Lessons + Bonus PDF

You claim: Points in the direction of greatest increase of a function.

Why? It can also point in the direction of greatest decrease of a function.

11. John Gabriel

Awesome, glad it worked for you

23. John Gabriel

increase will point below the middle of the

24. John Gabriel

25. John Gabriel

35. wali khan

37. j.sathish kumar

41. Ryan Johnson

43. Ranjeet Kumar

44. Shakeel Ahmed

48. aradhita chattopadhyay

thanks a lot for the wonderful explanation!!!

62. Mark Soric

72. bob clear

79. Nick Pellatz

90. Vitor P.F.

maximum gradient? should we choose (3,4,5) coordinates?

111. Patel Ankit

Consider the directional derivative, f_u.

116. beant singh

your space could feel a different push (strength and

would not gain height if you were moving, aka the

126. Baljeet Kaur

130. Kevin @ http://kldavenport.com

Thanks and keep up the good work Kalid!

136. Qammar Abbas

137. ATHUL P ANAND

Another explanation that I posted on reddit:

should have a lower-case delta ( instead of d

149. Amir sultan

150. Taps patel

You might also like