1 INTRODUCTION

1
2 INTELLIGENT AGENTS

function TABLE -D RIVEN -AGENT( 



) returns an action
static: 


 , a sequence, initially empty
 
, a table of actions, indexed by percept sequences, initially fully specified

append 


to the end of 



  L OOKUP( 


 ,  
)
return  

Figure 2.8

function R EFLEX -VACUUM -AGENT( [    ,    !" ]) returns an action

if    !" = # $ % then return & !'

else if (   = ) then return * (+,
else if (   = - then return .
0/

Figure 2.10

function S IMPLE -R EFLEX -AGENT( 



) returns an action
static: !1
 , a set of condition–action rules
   
2 I NTERPRET-I NPUT( 
 

)
!1
2 RULE -M ATCH(  3  
, !"
 )
  RULE -ACTION[ !1
]
return  

Figure 2.13

2
 3

function R EFLEX -AGENT-W ITH -S TATE( 



) returns an action
static:  3  
, a description of the current world state
!1
 , a set of condition–action rules
  , the most recent action, initially none
   
2 U PDATE -S TATE(  3  
,   , 


)
!1
2 RULE -M ATCH(  3  
, !"
 )
  RULE -ACTION[ !1
]
return  

Figure 2.16
3 SOLVING PROBLEMS BY
SEARCHING

function S IMPLE -P ROBLEM -S OLVING -AGENT( 

4 ) returns an action

inputs: 

4 , a percept
static: 
5 , an action sequence, initially empty
 3  
, some description of the current world state
+  , a goal, initially null


6 , a problem formulation
   
2 U PDATE -S TATE(  3  
, 
 

)
if 
5 is empty then do
+  F ORMULATE -G OAL(    
)
4 
67 F ORMULATE -P ROBLEM(  3  
, +  )

58 S EARCH(

6 )
  F IRST( 
5 )

59 R EST( 
5 )
return  

Figure 3.2

function T REE -S EARCH(


6 ,    
3+% ) returns a solution, or failure
initialize the search tree using the initial state of

6
loop do
if there are no candidates for expansion then return failure
choose a leaf node for expansion according to    
3+%
if the node contains a goal state then return the corresponding solution
else expand the node and add the resulting nodes to the search tree

Figure 3.9

4
 5

function T REE -S EARCH(


6;:0/$1+
) returns a solution, or failure
/0+1
< I NSERT(M AKE -N ODE(I NITIAL -S TATE[

6 ]), /$1+
)
loop do
if E MPTY ?( /$+1
) then return failure

=
2 R EMOVE -F IRST( /0+1
)
if G OAL -T EST[

6 ] applied to S TATE[ >=
] succeeds
then return S OLUTION( 
=
)
/$1+
< I NSERT-A LL(E XPAND( 
=
,

6 ), /$1+
)

function E XPAND( >=
:


6 ) returns a set of nodes
!
? the empty set
for each @   , 
!1 A in S UCCESSOR -F N[

6 ](S TATE[ >=
]) do
B a new N ODE
S TATE[  ] C
!1
PARENT-N ODE[  ] D>=

ACTION[  ] E  
PATH -C OST[  ]  PATH -C OST[ 
=
] + S TEP -C OST( >=
,   ,  )
D EPTH[  ]  D EPTH[ 
=
] + 1
add  to !

return !


Figure 3.12

function D EPTH -L IMITED -S EARCH( 4 
6 ,  06F$ ) returns a solution, or failure/cutoff

return R ECURSIVE -DLS(M AKE -N ODE(I NITIAL -S TATE[

6 ]),

6 ,  $6F$ )

function R ECURSIVE -DLS( >=
,


6 ,  $6F$ ) returns a solution, or failure/cutoff
!1 G !1
=H2 false
if G OAL -T EST[

6 ](S TATE[ >=
]) then return S OLUTION( 
=
)
else if D EPTH[ >=
] =  06F$ then return !" 3IG
else for each !
  in E XPAND( 
=
,

6 ) do

!1J 4 R ECURSIVE -DLS( !
 ,

6 ,  $6F$ )
if 
!1 = !1 IG then !1 IG !1
=H2 true
else if 
!1 LM K / $ !1
then return 
 !"
if !1 IG !1
=H then return !" 3IG else return / $ !1

Figure 3.17
6 Chapter 3. Solving Problems by Searching

function I TERATIVE -D EEPENING -S EARCH(


6 ) returns a solution, or failure
inputs:

6 , a problem

for =

0,N 0 to O do

!1J 4 D EPTH -L IMITED -S EARCH(

6 , =
4 , )
if 
!1 LM K cutoff then return 
!1J

Figure 3.19

function G RAPH -S EARCH(


6P:0/$1+
) returns a solution, or failure
(
=Q an empty set
/0+1
< I NSERT(M AKE -N ODE(I NITIAL -S TATE[

6 ]), /$1+
)
loop do
if E MPTY ?( /$+1
) then return failure

=
2 R EMOVE -F IRST( /0+1
)
if G OAL -T EST[

6 ](S TATE[ 
=
]) then return S OLUTION( >=
)
if S TATE[ 
=
] is not in (
= then
add S TATE[ >=
] to (
=
/$1+
< I NSERT-A LL(E XPAND( 
=
, 4 
6 ), /$1+
)

Figure 3.25
 INFORMED SEARCH AND
4 EXPLORATION

function R ECURSIVE -B EST-F IRST-S EARCH(


6 ) returns a solution, or failure
RBFS(

6 , M AKE -N ODE(I NITIAL -S TATE[

6 ]), O )

function RBFS(

6 , >=
, /  $6F0 ) returns a solution, or failure and a new R -cost limit
if G OAL -T EST[

6 ](  3  
) then return >=

!
? E XPAND( 
=
,

6 )
if  !4 
 is empty then return / $ !"
, O
for each  in !
 do
R [s] TSVUWYX0Z>XI[]\_^]XI[: R2` 
=
a0[
repeat

 4 the lowest R -value node in !

if R2` 
 acbC/ J$6F$ then return / $J!1
, R [ 
 ]
J 
>  $de
2 the second-lowest R -value among !


!1J , R [ 
 ]  RBFS( 4 
6 , 
 , Sgf(h?X/  06F$ :i  

 $d
[ )
if 
!1 LM K / $ !1
then return 
 !"

Figure 4.6

function H ILL -C LIMBING(


6 ) returns a state that is a local maximum
inputs:

6 , a problem
local variables: !1
4 , a node
>
(+,"  , a node
!1
 4 M AKE -N ODE(I NITIAL -S TATE[

6 ])
loop do
>
(+," 8
a highest-valued successor of !1

if VALUE[neighbor] j VALUE[current] then return S TATE[ !1
4 ]
!1
4 4DY
+, 

Figure 4.13

7
8 Chapter 4. Informed Search and Exploration

function S IMULATED -A NNEALING(


6 , ,
=!1
) returns a solution state
inputs:

6 , a problem
,
=!1
, a mapping from time to “temperature”
local variables: !1
4 , a node
>
k , a node
l
, a “temperature” controlling the probability of downward steps
!1
 4 M AKE -N ODE(I NITIAL -S TATE[

6 ])
for 4 1 to O do
l C,
=!"

[ ]
l
if = 0 then return !"
4
>
k
 selected successor of !"
4
mon  a randomly >
 


k !"
4 ]
mgn b VALUE[ !1]
–4V 4
D

ALUE[
Y


k
if 0 then
else !1
4 4DY
k only with probability piq2rBsIt

Figure 4.17

function G ENETIC -A LGORITHM( "

!1   , F ITNESS -F N) returns an individual
inputs: "
!1   , a set of individuals
F ITNESS -F N, a function that measures the fitness of an individual

repeat
>
u "
!1(   empty set
loop for  from 1 to S IZE( "
!1   ) do
kv R ANDOM -S ELECTION( "4!"(   , F ITNESS -F N)
%w R ANDOM -S ELECTION( "
!1(   , F ITNESS -F N)
,$=Q R EPRODUCE( k , % )
if (small random probability) then ,$(=v M UTATE( ,0(= )
add ,$(= to Y
u "
!1(  
"
!1  xC>
u "
!1(  
until some individual is fit enough, or enough time has elapsed
return the best individual in "
!1   , according to F ITNESS -F N

function R EPRODUCE( k , % ) returns an individual

inputs: k , % , parent individuals
 L ENGTH( k )
< random number from 1 to 
return A PPEND(S UBSTRING( k , 1,  ), S UBSTRING( % , y\{z ,  ))

Figure 4.21
 9

function O NLINE -DFS-AGENT( | ) returns an action

inputs:  | , a percept that identifies the current state
static: 
!1 , a table, indexed by action and state, initially empty
!1Y
k
(
= , a table that lists, for each visited state, the actions not yet tried
!1Y e'  'e
= , a table that lists, for each visited state, the backtracks not yet tried
 , , the previous state and action, initially null

if G OAL -T EST(  | ) then return  

if  | is a new state then !1Y
k
(
= [  | ]  ACTIONS(  | )
if  is not null then do

!1J [ ,  ] D |
add  to the front of !1Y e'  'e
= [  | ]
if
k4
= [  | ] is empty then
1
! >

if !1Y e'  'e
= [  | ] is empty then return  
else w an action  such that 
!1 [  ,  | ] = P OP( !1Y e'  'e
= [  | ])
else L P OP( !1Y
k
(
= [  | ])
<D |
return

Figure 4.25

function LRTA*-AGENT(  | ) returns an action

inputs:  | , a percept that identifies the current state
static: 
!1 , a table, indexed by action and state, initially empty
}
, a table of cost estimates indexed by state, initially empty
 , , the previous state and action, initially null

if G OAL -T EST(  | ) then return  

} }
if  | is a new state (not in ) then [  | ] ~^ (  | )
unless is null

!1J  D |
}   [ , ] Sofh }
[ ] € LRTA*-C OST(  ,  , 
 !" [  ,  ], )
ACTIONS ‚Iƒ
 an action  in ACTIONS( | ) that minimizes LRTA*-C OST( | ,  , 
!1 [  , | ], } )
L
<D |
return
}
function LRTA*-C OST(  , ,  | , ) returns a cost estimate
if  | is undefined then return ^?X3„i[
else return … X3„e: †
:„ | [g\ˆ‡ ` „ | a

Figure 4.29
 CONSTRAINT
5 SATISFACTION
PROBLEMS

function BACKTRACKING -S EARCH(  ) returns a solution, or failure

return R ECURSIVE -BACKTRACKING( ‰9Š ,  )

function R ECURSIVE -BACKTRACKING(  (+e46g
4 , 3 ) returns a solution, or failure

if (+e6g
4 is complete then return  (+e46g
4
d 8 S ELECT-U NASSIGNED -VARIABLE(VARIABLES[ 3 ], (+e6g
4 ,  )
for each d J!4
in O RDER -D OMAIN -VALUES( d  ,  +46g
4 ,  ) do
if d J!4
is consistent with  +46g
4 according to C ONSTRAINTS[ 3 ] then
add ‰ d  = d  !4
Š to (+e6g


!1J 4 R ECURSIVE -BACKTRACKING( (+e6g
 , 3 )
if 
!1J LM K / $J!1
then return 
!1
remove ‰ d  = d  !4
Š from (+46g

return / $J!1

Figure 5.4

10
 11

function AC-3( 3 ) returns the CSP, possibly with reduced domains

inputs: 3 , a binary CSP with variables ‰i‹Œ : ‹gŽ :: ‹o‘
Š
local variables: 5!4
!4
, a queue of arcs, initially all the arcs in 3

while 5!4
!

is not empty do
X ‹o’ : ‹F“ [4 R EMOVE -F IRST( 5!4
!4
)
if R EMOVE -I NCONSISTENT-VALUES(‹ ’ : ‹ “ ) then
for each ‹V” in N EIGHBORS[‹o’ ] do
add (‹c” : ‹ ’ ) to 5!4
!4

function R EMOVE -I NCONSISTENT-VALUES( ‹ ’ : ‹ “ ) returns true iff we remove a value


6od
=v•/  

for each k in D OMAIN[‹ ’ ] do
if no value – in D OMAIN[‹ “ ] allows ( k , % ) to satisfy the constraint between ‹ ’ and ‹ “
then delete k from D OMAIN[‹ ’ ]; 
6od
=vD !4

return 
6od
=

Figure 5.9

function M IN -C ONFLICTS(  , 6o ik  


 ) returns a solution or failure
inputs: 3 , a constraint satisfaction problem
6o ik  

 , the number of steps allowed before giving up
!1 
 4 an initial complete assignment for 
for  = 1 to 6o ik  

 do
if !1
 is a solution for  then return !1
4
d 8 a randomly chosen, conflicted variable from VARIABLES[  ]
d  !

< the value d for d  that minimizes C ONFLICTS( d  , d , !1
4 , 3 )
set d  M d J!4
in !"
4
return / $J!1

Figure 5.11
6 ADVERSARIAL SEARCH

function M INIMAX -D ECISION(    
) returns —  

inputs:    
, current state in game
dw M AX -VALUE(state)
return the   in S UCCESSORS(  3  
) with value d

function M AX -VALUE(    
) returns F!1 $J$ %˜d J!4

if T ERMINAL -T EST(    
) then return U TILITY(    
)
dwC™ O
for 4: in S UCCESSORS(  3  
) do
dw M AX(v, M IN -VALUE(  ))
return d

function M IN -VALUE(  3  
) returns x!1 $ 0 %˜d J!4

if T ERMINAL -T EST(    
) then return U TILITY(    
)
dw O
for 4: in S UCCESSORS(  3  
) do
dw M IN(v, M AX -VALUE(  ))
return d

Figure 6.4

12
 13

function A LPHA -B ETA -S EARCH(    
) returns an action

inputs:    
, current state in game
dw M AX -VALUE(    
, ™ O , \ O )
return the   in S UCCESSORS(  3  
) with value d

function M AX -VALUE(    
, š , › ) returns x!" $J$ %Fd  !4

inputs:    
, current state in game
š , the value of the best alternative for MAX along the path to    

› , the value of the best alternative for MIN along the path to    

if T ERMINAL -T EST(    
) then return U TILITY(    
)
dwC™ O
for 4: in S UCCESSORS(  3  
) do
dw M AX(v, M IN -VALUE(  , š , › ))
if œžŸ› then return d
š  M AX( š , v)
return d

function M IN -VALUE(  3  
, š , › ) returns F!1 $ $ %˜d  !

inputs:    
, current state in game
š , the value of the best alternative for MAX along the path to    

› , the value of the best alternative for MIN along the path to    

if T ERMINAL -T EST(    
) then return U TILITY(    
)
dw~\ O
for 4: in S UCCESSORS(  3  
) do
dw M IN(v, M AX -VALUE(  , š , › ))
if œžj š then return d
›  M IN(› , v)
return d

Figure 6.9
7 LOGICAL AGENTS

function KB-AGENT( 



) returns an  
static: ¡¢- , a knowledge base
, a counter, initially 0, indicating time

T ELL( ¡¢- , M AKE -P ERCEPT-S ENTENCE( 



, ))
  A SK( ¡¢- , M AKE -ACTION -Q UERY( ))
T ELL( ¡¢- , M AKE -ACTION -S ENTENCE(   , ))

D + 1
return  

Figure 7.2

function TT-E NTAILS ?( ¡¢- , š ) returns !4
or / J

inputs: ¡¢- , the knowledge base, a sentence in propositional logic
š , the query, a sentence in propositional logic
%6g  2 a list of the proposition symbols in ¡¢- and š
return TT-C HECK -A LL( ¡N- , š , %6g   , ` a )

function TT-C HECK -A LL( ¡¢- , š , %6g   , 6o=
 ) returns !4
or /  

if E MPTY ?( %6g   ) then
if PL-T RUE ?( ¡N- , 6o=
 ) then return PL-T RUE ?( š , 6o=
 )
else return !4

else do
£ 
F IRST( %6g   ); 
 4 R EST( %6g   )
£
return TT-C HECK -A LL( ¡¢- , š , 
 , E XTEND( : !4
e:6o=
 ) and
 


 £
TT-C HECK -A LL( ¡¢- , š , , E XTEND( :0/ J
:6o=
 )

Figure 7.12

14
 15

function PL-R ESOLUTION( ¡¢- , š ) returns !4
or /  

inputs: ¡¢- , the knowledge base, a sentence in propositional logic
š , the query, a sentence in propositional logic
( !1
B the set of clauses in the CNF representation of ¡¢-_¤P¥9š
Y
u¦ ‰9Š
loop do
for each § ’ , § “ in ( !1
 do

Jd
4 B PL-R ESOLVE( § ’ , § “ )
if 
 de
4  contains the empty clause then return !4

Y
u¦DY
u—¨©
 de
 
if u«ª•( !1
 then return /  

 Y 


 !"
<¬( !1
?¨cY
u

Figure 7.15

function PL-FC-E NTAILS ?( ¡¢- , 5 ) returns !4
or /  

inputs: ¡¢- , the knowledge base, a set of propositional Horn clauses
5 , the query, a proposition symbol
local variables:  !14 , a table, indexed by clause, initially the number of premises
$/

= , a table, indexed by symbol, each entry initially / J

+1

= , a list of symbols, initially the symbols known to be true in ¡¢-

while +
>=e is not empty do

­ P OP( +1
>= )
unless 0/

= [ ] do
$/

= [ ] C !4

for each Horn clause  in whose premise appears do
decrement !14 [  ]
if !" [  ] = 0 then do
if H EAD[  ] = 5 then return !4

P USH(H EAD[  ], +1
>= )
return 
/ 
 


Figure 7.18
16 Chapter 7. Logical Agents

function DPLL-S ATISFIABLE ?(  ) returns !4
or /  

inputs:  , a sentence in propositional logic
( !1
B the set of clauses in the CNF representation of 
%6g  2 a list of the proposition symbols in 
return DPLL(  !1
 ,  %6g J , ` a )

function DPLL( ( !1
 , %6g   , 6o=
 ) returns !

or / J

if every clause in ( !1
 is true in 6o=
 then return !4

if some clause in  !1
 is false in 6o=
 then return / J

® d J!4
2
, F IND -P URE -S YMBOL( %6gJ ,  !"
 , 6o=
 )
® ® £
if is non-null then return DPLL( ( !1
 ,  %6g J – , E XTEND( : d J!4
:6o=
 )
® d J!4
2 
 (
 
1
! 
 

 o
6 
 
= 


, F IND -U NIT-C LAUSE( , )
® ® £
if is non-null then return DPLL( ( !1
 ,  %6g J – , E XTEND( : d J!4
:6o=
 )
®  
 
% g
6  
    


 4
 
 
% g
6 
 
 J
 
F IRST( ); R EST( )
®
return DPLL(  !1
 , 
 , E XTEND( , !4
, 6o=
 )) or
®
DPLL(  !1
 , 
 , E XTEND( , /  
, 6o=
 ))

Figure 7.21

function WALK SAT( ( !1
 , , 6o ik ¯y

 ) returns a satisfying model or / $J!1

inputs: ( !1
 , a set of clauses in propositional logic
, the probability of choosing to do a “random walk” move, typically around 0.5
6o ik ¯y
 , number of flips allowed before giving up
6o=
e a random assignment of !4
//  
to the symbols in ( !1

for  M z to 6o ik ¯y 4 do
if 6o=
 satisfies  !1
 then return 6o=

 !"
2 a randomly selected clause from ( !1
 that is false in 6o=

with probability flip the value in 6o=
 of a randomly selected symbol from ( !1

else flip whichever symbol in  !1
maximizes the number of satisfied clauses
return / $J!1

Figure 7.23
 17

function PL-W UMPUS -AGENT( 
 


) returns an  
inputs: 

4 , a list, [  
>, , 

°i
, +e $  
 ]
static: ¡¢- , a knowledge base, initially containing the “physics” of the wumpus world
k , % , 3
4 3   , the agent’s position (initially 1,1) and orientation (initially (+, )
di$$ 
= , an array indicating which squares have been visited, initially /  

  , the agent’s most recent action, initially null

(  , an action sequence, initially empty

update k , % , 
4    , di$$ 
= based on e 

if  

, then T ELL( ¡¢- , ±?²³ ´ ) else T ELL( ¡¢- , ¥—±]²³ ´ )
if 

°i
then T ELL( ¡¢- , µN²³ ´ ) else T ELL( ¡¢- , ¥—µ¢²³ ´ )
if +J$  
 then  ­~+ 
else if 4  is nonempty then   P OP(
(  )
£
else if for some fringe square [  ,¶ ], A SK( ¡¢- , X ¥ ’ ³ “ ¤T¥—· ’ ³ “ [ ) is !4
or
£
for some fringe square [ ,¶ ], A SK( ¡¢- , ’ ³ “¸ · ’ ³ “ [ ) is / J
then do
 X
4  A*-G RAPH -S EARCH(ROUTE -P ROBLEM([ k , % ], 3
4 3   , [  ,¶ ], di$$ 
= ))
e ­ P OP(
(  )
else   a randomly chosen move
return  

Figure 7.26
8 FIRST-ORDER LOGIC

18
9 INFERENCE IN
FIRST-ORDER LOGIC

function U NIFY( k , % , ¹ ) returns a substitution to make k and % identical

inputs: k , a variable, constant, list, or compound
% , a variable, constant, list, or compound
¹ , the substitution built up so far (optional, defaults to empty)
if ¹ = failure then return failure
else if k = % then return ¹
else if VARIABLE ?( k ) then return U NIFY-VAR( k , % , ¹ )
else if VARIABLE ?( % ) then return U NIFY-VAR( % , k , ¹ )
else if C OMPOUND ?( k ) and C OMPOUND ?( % ) then
return U NIFY(A RGS[ k ], A RGS[ % ], U NIFY(O P[ k ], O P[ % ], ¹ ))
else if L IST ?( k ) and L IST ?( % ) then
return U NIFY(R EST[ k ], R EST[ % ], U NIFY(F IRST[ k ], F IRST[ % ], ¹ ))
else return failure

function U NIFY-VAR( d  , k , ¹ ) returns a substitution

inputs: d  , a variable
k , any expression
¹ , the substitution built up so far
if ‰ d º1d  Š¦»7¹ then return U NIFY( d  , k , ¹ )
else if ‰¼ º"d  Š¦»7¹ then return U NIFY( d  , d  , ¹ )
else if O CCUR -C HECK ?( d  , k ) then return failure
else return add ‰ d  / k Š to ¹

Figure 9.2

19
20 Chapter 9. Inference in First-Order Logic

function FOL-FC-A SK( ¡N- , š ) returns a substitution or / J

inputs: ¡¢- , the knowledge base, a set of first-order definite clauses
š , the query, an atomic sentence
local variables: >
u , the new sentences inferred on each iteration

repeat until Y
u is empty

>
u— ‰9Š
for each sentence  in ¡N- do
X Œ<¤  ¤ ‘•½ 5[4 S TANDARDIZE -A PART(  )
for each ¹ such that S UBST( ¹ , Œ ¤  ¤ ‘ ) = S UBST( ¹ , Œ| ¤  ¤ ‘ | )
for some Œ| ::I ‘ | in ¡N-
5 |  S UBST(¹ , 5 )
if 5 | is not a renaming of some sentence already in ¡¢- or >
u then do
add 5 | to Y
u
¾ 
(5 | , š )
¾ U NIFY ¾
if is not $ then return

/ 

add Y
u to ¡¢-
return /  

Figure 9.5

function FOL-BC-A SK( ¡¢- , + J , ¹ ) returns a set of substitutions

inputs: ¡¢- , a knowledge base
+   , a list of conjuncts forming a query ( ¹ already applied)
¹ , the current substitution, initially the empty substitution ‰9Š
local variables: 4uy
 , a set of substitutions, initially empty

if +   is empty then return ‰¹Š

5 |  S UBST( ¹ , F IRST( +   ))
for each sentence  in ¡¢- where S TANDARDIZE -A PART(  ) = X ŒP¤  ¤ ‘ž½À¿ [
and ¹ |  5 5 |
U NIFY( , ) succeeds
>
u + JB ` Œ ::I ‘BÁ R EST( +   )a
4uy
B FOL-BC-A SK( ¡¢- , Y
u +   , C OMPOSE( ¹ | , ¹ )) ¨Â 4uy

return uy


Figure 9.9

procedure A PPEND( k , % , i° , 4 $4!   )

 $ G LOBAL -T RAIL -P OINTER()
if k = ` a and U NIFY( % , i° ) then C ALL( 4 $!4   )
R ESET-T RAIL(  $ )
L N EW-VARIABLE(); kà N EW-VARIABLE(); °y N EW-VARIABLE()
if U NIFY( ik , [ — k ]) and U NIFY( i° , [ — ° ]) then A PPEND( k , % , ° ,  $4!   )

Figure 9.12
 21

procedure OTTER(  , !" 
)

inputs:  , a set of support—clauses defining the problem (a global variable)
!" 
, background knowledge potentially relevant to the problem

repeat
clause  the lightest member of 
move ( !1
from  to !1 

P ROCESS(I NFER( ( !1
, !" 
),  )
until  = ` a or a refutation has been found

function I NFER( ( !1
, !1 
) returns clauses

resolve ( !1
with each member of !" 

return the resulting clauses after applying F ILTER

procedure P ROCESS( ( !1
 ,  )

for each ( !1
in ( !1
 do

 !"
2 S IMPLIFY( ( !1
)
merge identical literals
discard clause if it is a tautology
< [ ( !1
—  ]
if ( !1
has no literals then a refutation has been found
if ( !1
has one literal then look for unit refutation

Figure 9.19
10 KNOWLEDGE
REPRESENTATION

22
11 PLANNING

Ä $ X X : £ X £ :yÈ Å2¡ [

) § Œ &"ÅLÆ [ ¤Ç) X § Ž :yÈ Å2¡ [® ¤Ç) £ X Œ : &"Åw [
® Æ ( Y
¤É £) [ Ž
¤_Ê 

 
+ 4
 X §vŒ ¤ŸÊ [ 
3
 
+ 
 X §LŽ ¤ [ 
 
Y
 

X [
Œ ¤ X Ž
¦
¤ ) $" XÈ Å2¡ [ ¤É) 0" X &"ÅwÆ [[
Ë  X X :yÈ
) §vŒ Å2¡ [ ¤É) X §LŽ : &1ÅLÆ [[
)  2X .  e=]X … :>Ì]:<†"[: ® ( >
X(Ì
[
P RECOND : ) X … :2†1[ ¤Ç) X(Ì]:<†"[ ¤ˆÊ +X … [ ¤ ¤Ç) $" X†"[

X 2
: "
† [ Ä 2X :
Ì
[[
E FFECT: ¥¦) … ¤ …
)  2XÍB4( Ä =]X … :
Ì]:2†"[: ® ( Y
X(Ì>[
P RECOND : 2X … :>Ì
[ ¤Ç) X(Ì]:<†1[ ¤ˆÊ +X … [ ¤ ¤Ç) $" X†"[

X 2
: "
† [ Ä 2X :
Ì
[[
E FFECT: ) … ¤T¥ …
)  2X Å  %>X(Ì]:>/6P:y 3[: ® ( >
X(Ì
[
P RECOND : ) X(Ì?:
/ 6c[ ¤ ¤Ç) $" X(/6V[ ¤Ç) $" XI [
E FFECT: ¥¦) 
(
X ]
Ì >
: 
/ 
 
 V
6 [ ¤É) X(Ì]:y 3[[

Figure 11.3

23
24 Chapter 11. Planning

Ä  $ X X (  : ) k
i[ ¤É) X & " 
: l !1'"[[

Ë  I) X Å X
) & " 
e: ) k
[[ l
)  2X *
6ode
eX & " 
: !""'"[ ,
l
P RECOND : ) X & " 
: !1'"[

X "   e

: l !1"'[ Ë
E FFECT: ¥¦) & ¤Ç) X & " 
: !1>=[[
) 
 

 
 2
 X * 

o
6 
 e
d e

X Å 
 

: ) 
k 
 

[ ,
P RECOND : ) X Å   : ) k
i[
Ë
E FFECT: ¥¦) X Å (  : ) k
[ ¤Ç) X Å (  : !1>=[[

 

 
 2
 X ® !1 2X " 
: k
i[
) Æ & ) Ë ,
P RECOND : ) X & " 
: !1>=4[ ¤É¥¦) X Å (  : ) k
[
Ë
E FFECT: ¥¦) X & " 
:  !"
=4[ ¤É) X & " 
e: ) k
[[
) 
 

 
 2
 X .


d
Æ 
d 


 4
 (
 
+ 
, ,
P RECOND :
Ë
: ) k
i[ ¤T¥¦)  X & " 
: l !""'"[
E FFECT: ¥¦)  X & " 
:  !"
=4[ ¤T¥¦)  X & "  
Ë
¤;¥¦)  X Å   :  !"
=4[ ¤T¥¦)  X Å   : ) k
i[[

Figure 11.5

Ä $ X <XÎ: l 
[ l l

Æ ¤ˆÆ 2X µ : 
i[ ¤ˆÆ 2X § : 
i[
¤É- (
 
 
 >
' 
X N
Î [ ¤É- (
 
 
 >
' X µ [ ¤É- (
 
 
 >
' X § [
ˆ
¤ Ê 
 "XÎN[ ¤ŸÊ 
1X µ [ ¤ˆÊ 
 "X § [[
Ë  X 2XÎ: [
Æ µ ¤ˆÆ <X µ : § [[
) 
 

 
 2
 
X Ð
Ï 
 e
d 

3
X i
Ñ : ¼ : – [:
P RECOND : Æ 2
 3
X Ñi: ¼ [ ¤ŸÊ 
1X3Ñ[ ¤ˆÊ 
 "X – [ ¤É- '>X3Ñ[ ¤
X3Ñ{M K ¼ [ ¤ X3ÑM K – [ ¤ X ¼ M K – [ ,
E FFECT: Æ <X3Ñi: – [ ¤ˆÊ 
"X ¼ [ ¤T¥_Æ 2X3Ñi: ¼ [ ¤É¥_Ê 
1X – [[
l l
) 2XÏÐde
 
X3Ñi: ¼ [:


P RECOND : Æ 2X3Ñi: ¼ [ ¤ŸÊ 
1X3Ñ[ ¤É- ('>X3Ñ[ ¤ X3Ñ{M K ¼ [ ,
l
E FFECT: Æ <X3Ñi: 
[ ¤ŸÊ 
1X ¼ [ ¤T¥_Æ <X3Ñi: ¼ [[

Figure 11.7
 25

Ä  $ X X (  : ) k
i[ ¤É) X & " 
: l !1'"[[

Ë  I) X Å X
) & " 
e: ) k
[[ l
)  2X *
6ode
eX & " 
: !""'"[ ,
l
P RECOND : ) X & " 
: !1'"[
l Ë
E FFECT: ¥¦) X & " 
e: !1"'[ ¤Ç) X & " 
: !1>=[[
) 
 

 
 2
 X * 

o
6 
 e
d e

X Å 
 

: ) 
k 
 

[ ,
P RECOND : ) X Å   : ) k
i[
Ë
E FFECT: ¥¦) X Å (  : ) k
[ ¤Ç) X Å (  : !1>=[[
®
)  2X !1 Æ 2X & " 
: ) kË 
i[ ,
P RECOND : ) X & " 
: !1>=4[ ¤É¥¦) X Å (  : ) k
[
Ë
E FFECT: ¥¦) X & " 
:  !"
=4[ ¤Ç) X & " 
e: ) k
i[[
) 
 

 
 2
 X .


d
Æ 
d 


 4
 (
 
+ 
, ,
P RECOND :
Ë
: ) k
i[ ¤T¥¦)  X & " 
: l !""'"[
E FFECT: ¥¦)  X & " 
:  !"
=4[ ¤É¥¦)  X & "  
Ë
¤;¥¦)  X Å   :  !"
=4[ ¤T¥¦)  X Å (  : ) k
i[[

Figure 11.11

Ä  $ X } de
eX '
i[[
Ë  X } d
XÊ '
[ n  
2X i'
i[[
n Ê ¤ Ê
)  2X  X Ê } i'
i[
P RECOND : d
X Ê '
[
} de
X 'e
i[ n  
2X 'e
[
E FFECT: ¥ Ê ¤ Ê )
)  2X - i'
X Ê } '
[
P RECOND : ¥ d
X '
[
} de
X 'e
iÊ [
E FFECT: Ê

Figure 11.16

function G RAPHPLAN(

6 ) returns solution or failure
+e 4,v I NITIAL -P LANNING -G RAPH(

6 )
+  < G OALS[ 4 
6 ]
loop do
if + J all non-mutex in last level of + 4, then do
 !1  E XTRACT-S OLUTION( +e 4, , +   , L ENGTH( +e , ))
if  !" ÐM K / $ !1
then return J!1 
else if N O -S OLUTION -P OSSIBLE( + 4, ) then return / $ !1

+ 4,Ã E XPAND -G RAPH( + 4, ,

6 )

Figure 11.19
26 Chapter 11. Planning

l˜Ò2ÓÔ
function SAT PLAN(

6 , ) returns solution or failure
inputs:

6 , a planning problem
l˜Ò2ÓÔ
, an upper limit for plan length
l l˜Ò2ÓÔ
for = 0 to do
/ , 6o 
$1+Ã T RANSLATE -T O -SAT(

6 , l )
(+46g
 4 SAT-S OLVER( / )
if  +46g
4 is not null then
return E XTRACT-S OLUTION(  (+e46g
4 , 6o 
$+ )
return / $J!1

Figure 11.22
 PLANNING AND ACTING
12 IN THE REAL WORLD

ÕÖ]×3ØiÙÚwÛ>ÜÝÝ×Ý"ÙÞFßà9áâÚyÛ>ÜÝÝ×Ý"ÙÞ­ãà
áåäNÖ4æ1×ÖBç
ÙègßéÞFßéêë4à9áÂäÃÖ
æ1×Ö<ç
ÙèFãéÞ­ãéì1ë4à
áîíïÛ>ççðJÝÙIñ7ßéÞFßéê1ë4à9áîíÐÛ>ççið(ÝÙIñ_ãéÞ¢ãéòeóàà
ôõ Ü1ð ÙIö õ Ö<ç4ÙÞxßà9áÂö õ ÖBç
ÙÞ­ãàà
÷xø Ø3× õ ÖLÙ ÷xùù äNÖ
æ×Ö<ç4Ùú1éûeéüýàé
äNÖ
æ1×Ö<ç
Ùú1éûeéþ>à2áâÚyÛ>ÜÝÝ×ÝÙûà9áåÿQäNÖ
æ×Ö<çÕÖLÙûàé
äNÖ
æ×Ö<çÕÖLÙûà9áå
ö Ü1Ø3× õ ÖLÙþ
àà
P RECOND :
÷xø Ø3× Eõ ÖL Ù ÷xùù íÐÛ>ççiðJÝÙ Véûàé íÐÛ>ççiðJÝ4Ù géûeéþ>à9á«ÚwÛ>ÜÝÝ×ÝÙûàé
FFECT:

íÐÛ>ççiðJÝ ÃÖLÙûà9áÂ
ö Ü1Ø3× ÖwÙþ>àà

P RECONDõ :
÷xø Ø3× Eõ ÖL ÙIÕÖY Ý ]ç ø ØiÙûàé äNÖ4æ1×ÖBçÕÖQÙûà2áTíÐÛ>ççiðJÝ ÃÖLÙûà9áâÚyÛ>ÜÝÝ×ÝÙûàé
FFECT:

ö õ Ö<ç4Ùûà2áÂ
ö Ü1Ø3× õ ÖLÙòëàà

P RECOND :
E FFECT:

Figure 12.2

27
28 Chapter 12. Planning and Acting in the Real World

ÕÖ]×ØÙÚyÛ>ÜÝÝ×Ý"ÙÞFßà9áâÚyÛ>ÜÝÝ×ÝÙÞ¢ãeà
áåäNÖ4æ1×ÖBç
ÙègßéÞFßéêë4à9áÂäÃÖ
æ1×Ö<ç
ÙèFãéÞ­ãéì1ë4à
áîíïÛ>ççðJÝÙIñ7ßéÞFßéê1ë4à9áîíÐÛ>ççið(ÝÙIñ_ãéÞ¢ãéòeóà
áåäNÖ4æ1×ÖBç
õ ×ÝØ3Ý4Ùòà2áTíÐÛ>ççið BØIÜ1Ø3× õ Ö]ÝÙòà9áåÕÖ]Ý ]ç ø Ø õ  ÝÙ àà
ôõ Ü1ð ÙIö õ Ö<ç4ÙÞxßià2áÂö õ Ö<ç4ÙÞ­ãàà
÷xø Ø3× õ ÖwÙ ÷xùù äNÖ
æ×Ö<ç4Ùú1éûeéü àé
äNÖ
æ1×Ö<ç
Ùú1éûeéþ>à2áâÚwÛ>ÜÝÝ×Ý"Ùûà9áåÿQäNÖ
æ×Ö<çÕÖLÙûàé
äNÖ
æ×Ö<çÕÖLÙûà9áå
ö Ü1Ø3× õ ÖLÙþ
àé
P RECOND :
äNÖ
æ×Ö<ç
õ ×ÝØ3Ý4Ùòàà
E FFECT:
÷xùù íÐÛ>ççi:ðJÝÙ Véûàé
÷xø Ø3× õ ÖwRÙ ESOURCE
íÐÛ>ççið(ÝÙ Véûeéþ
à9áâÚyÛ>ÜÝÝ×ÝÙûàé
íÐÛ>ççiðJÝ ÃÖQÙûà2áÂ
ö Ü1Ø3× õ ÖwÙþ>àé
P RECOND :
E FFECT:
íÐÛ>ççð BØIÜ1Ø3× õ ÖYÝ4Ùòàà
ÕÖY Ý ]ç ø ØiÙûàé
÷xø Ø3× õ ÖwRÙIESOURCE :
äNÖ
æ1×Ö<çÕÖLÙûà9áTíïÛ>ççðJÝ ÃÖLÙûàé
ö Ö<ç4Ùûà2áÂ
ö Ü1Ø3× õ ÖLÙòëàé
P RECOND :õ
ÕÖ] Ý ]ç ø Ø õ Ý"Ùòàà
E FFECT:
R ESOURCE :

Figure 12.5

)  2X - Ë 1! % .  
=]: P RECOND :Ë ÏïY
%
: E FFECT: . >= ¤P¥ ÏÐY
%1[
)  2X
 .  <: P RECOND : = Ê 
=$ : E FFECT: ÏÐY
% ¤ ÏÐ $+ +1
[
} }
)  2X - $(=  !"
: P RECOND : . >=Y: E FFECT: !1
i[
1
!
Ë ® ®
)  2X } 

6F0 : P RECOND : . >=Y: E FFECT:
6F$ [
) 
 

 
 2
 X $
  
- 1
! $
 
 
= 

1
 : E FFECT: Ê 
 
 

 

 
[
®
6F0
)  2X Ê }  ! <: P RECOND ®
6F:$ [ ¤ˆÊ 4   :
E FFECT: 
 1
! 

- 1
! 0
  ¤T¥
® ÏÐ>
% ¤ } !1
- !"$ :
)  2X % - !10(=
": P RECOND } !1
:
E FFECT: ¥ Ð
Ï 
 Y
 

% ¤ ¤T¥_Ê    [
}
#
®  6Ã"
X - !"$(= !1Ë
: ® }
( 2X S TEPS e‰±< Œ 

6F$ : ±] Ž  $®
- !1$=
1:
± VÊ   
 4
 
 




! 
 

 
 B


: ±   % - !10(=
$ 4Š $ ,Y: 3 
  ± iŠ
O RDERINGS e‰& ±BŒ ±  ±  Å & ±]Ž :
L INKS e‰&    ™  !   ±BŒ : &   "$ ™ # ! %'&  ±  :
± ) Œ ( ™ %+*+! ,.- / ±  : ± 1 Ž 0 # ™ 2! /3*465+/ ±  : ± 87 # 9; ™ : !%'< 9 - = / ±  :
± 7 Å ™ 
# ! >
9 : % $
 4
 $
  Y
, : ± ? ™ @
"
 ! #  +
% & Å $
 
 0
 
 , Š [[

Figure 12.9
 29

function A ND -O R -G RAPH -S EARCH(


6 ) returns o >=$ > "
( <:L9/ $ !"

O R -S EARCH(I NITIAL -S TATE[

6 ],

6 , ` a )

function O R -S EARCH(    
,

6 , "  , ) returns o
=$ 
4 B:L9/ $J!1

if G OAL -T EST[

6 ](  3  
) then return the empty plan
if    
is on "  , then return / $J!1

for each   ,  3  

 in S UCCESSORS[

6 ](  3  
) do
4  A ND -S EARCH(  3  

 ,

6 , `  3  
Á "  ,"a )
if
( ÐM K / $ !"
then return `   Á
( >a
return / $J!1

function A ND -S EARCH(    

 ,


6 , "  , ) returns ˜

=$ > 
( <:L2/ 0 !1

for each  ’ in    

 do
4  ’  O R -S EARCH(  ’ ,

6 , "  0, )
if
(  = / $ !"
then return / $J!1

BA
return ` if  Πthen
(  Œ else if  Ž then
(  Ž else  if  ‘ ?Œ then
  ‘ ?Œ else
(  ‘ a
BA
Figure 12.14

function R EPLANNING -AGENT( 



) returns an  
static: ¡¢- , a knowledge base (includes action descriptions)

(  , a plan, initially ` a
u2,"

  , a plan, initially ` a
+  , a goal

T ELL( ¡¢- , M AKE -P ERCEPT-S ENTENCE( 



, ))
!1
 4 S TATE -D ESCRIPTION( ¡¢- , )
if 4  M ` a then
u2,"

( î
  P LANNER( !1
 , +  , ¡N- )
if P RECONDITIONS( F IRST( 4  )) not currently true in ¡¢- then
candidates  S ORT( u2,"

(  , ordered by distance to !1
 )
find state „ in  
==  
 such that
/ $J!1
xM K 
" $8 P LANNER( !1
4 , „ , ¡¢- )
 4 0!   the tail of u2,"

(  starting at „
u2,"

( î
  A PPEND( 
" $ ,  $4!   )
return P OP(
(  )

Figure 12.18
30 Chapter 12. Planning and Acting in the Real World

function C ONTINUOUS -POP-AGENT( 
 


) returns an  
static:
(  , a plan, initially with just &   , Å 0$ ,
 DC­ Æ (the default)
E FFECTS[ &   ] = U PDATE(E FFECTS[ & 3  ], 


)
R EMOVE -F LAW(
  ) // possibly updating  
return  

Figure 12.28

)Ä +
4 eXÎ: µ [
$ X ) XÎ: ` .
0/ : - 
 0>
a0[ ¤Ç) X µ : ` * (+, :C
 a0[ ¤

 ,$1+YX $: +, : 
 0>
a0[[ ®  Y
1XÎ: [ ®  Y
"X : ÎN[
Ë  ) X
 !1Y
=>X - $[ ` * X +1-
4 : :ECx
 a0¤ [[ µ ¤ µ
* } - ¤É) `¼
)  2X $ X +1
4 : - $[:
P RECOND : ) e
 ,$+YX - $: ` ¼ : – a0[ ¤É) X +1
4 : ` ¼ : – 0a [ ¤
®  Y
1X +1
 :I"  Y
[
¤T¥¦) X0"  >
: ` ¼ : – a0[
E FFECT: * 


1
! 
 Y

Y
= X - 
$
 
 
[ [
Ë
)  2X 4X +
4 : ` ¼ : – a0[:
P RECOND : ) X +1
4 : ` †4:9Ña0[:
E FFECT: ) X +1
 : ` ¼ : – a$[ ¤T¥¦) X +1
4 : ` †4:9Ñ a$[[

Figure 12.30
13 UNCERTAINTY

function DT-AGENT( 

4 ) returns an  

static: 
J
/  3  
, probabilistic beliefs about the current state of the world
  , the agent’s action

update 
J
0/  3  
based on   and 



calculate outcome probabilities for actions,
given action descriptions and current 
 3
0/  3  

select   with highest expected utility
given probabilities of outcomes and utility information
return  

Figure 13.2

F
function E NUMERATE -J OINT-A SK( , e, P) returns a distribution over F
F
inputs: , the query variable
e, observed values for variables E
P, a joint distribution on variables ‰i‹ïŠ ¨ E ¨ Y /* Y = hidden variables */

F F
Q X [4 a distribution over , initially empty
for each value ¼ ’ of do F
Q X ¼ ’ [4 E NUMERATE -J OINT(¼ ’ , e, Y, ` a , P)
return N ORMALIZE(Q X ‹ [ )

function E NUMERATE -J OINT(¼ , e, d  , d J!4
 , P) returns a real number

if E MPTY ?( d  ) then return P X ¼ : e :d  !4
[
G  F IRST( d  )
H
return ´ E NUMERATE -J OINT(¼ , e, R EST( d  ), ` –]Á d  !4
a , P)

Figure 13.6

31
 PROBABILISTIC
14 REASONING

F
function E NUMERATION -A SK( , e,  ) returns a distribution over F
F
inputs: , the query variable
e, observed values for variables E
 , a Bayes net with variables ‰i‹ÐŠ ¨ E ¨ Y /* Y = hidden variables */

F F
Q X [4 a distribution over , initially empty
for each value ¼ ’ of do F
extend e with value ¼ ’ for F
Q X ¼ ’ [4 E NUMERATE -A LL(VARS[  ], e)
return N ORMALIZE(Q X ‹ [ )

function E NUMERATE -A LL( d  , e) returns a real number

if E MPTY ?( d  ) then return 1.0
I  F IRST( d  )
I
if has value % in e
£ KJ LNM G PO
then return X –cÁ Ì4† p „X [[ E NUMERATE -A LL(R EST( d  ), e)
H £ X Ì
† KJ LM G PO

„ X 
[ [ E NUMERATE -A LL(R EST( d  ), e ´ )
else return ´ –gÁ
where e´ is e extended with
p G
M –

Figure 14.10

F
function E LIMINATION -A SK( , e,  ) returns a distribution over F
F
inputs: , the query variable
e, evidence specified as an event
 , a Bayesian network specifying joint distribution P X ‹cŒ :: ‹o‘ [
/  3< ` a ; d B R EVERSE(VARS[  ])
for each d  in d  do
/  B ` M AKE -FACTOR XId : e [ Á / e a
if d  is a hidden variable then /  3B S UM -O UT( d  , /  3 )
return N ORMALIZE(P OINTWISE -P RODUCT(/  3 ))

Figure 14.12

32
 33

function P RIOR -S AMPLE(  ) returns an event sampled from the prior specified by 
inputs:  , a Bayesian network specifying joint distribution P X ‹cŒ :: ‹o‘ [

x  an event with elements L

Q
for M z to do L J pLNM „X ‹
¼ ’  a random sample from P X ‹ ’ Á Ì4† ’ [[
return x

Figure 14.15

F C
function R EJECTION -S AMPLING( , e,  , ) returns an estimate of
£ X
‹ Á e[
F
inputs: , the query variable
e, evidence specified as an event
 , a Bayesian network
C
, the total number of samples to be generated
F
local variables: N, a vector of counts over , initially zero

for ¶ = 1 to do R
x  P RIOR -S AMPLE(  )
if x is consistent with e then
N[ k ]  N[ k ]+1 where k is the value of F in x
return N ORMALIZE(N[ ]) F
Figure 14.17

F C
function L IKELIHOOD -W EIGHTING( , e,  , ) returns an estimate of
£ X
‹ Á e[
F
inputs: , the query variable
e, evidence specified as an event
 , a Bayesian network
C
, the total number of samples to be generated
local variables: W, a vector of weighted counts over , initially zero F
for ¶ = 1 to do R
x, u— W EIGHTED -S AMPLE(  )
W ` k"a W ` k"a\7u where k is the value of F in x
return N ORMALIZE(W ` a ) F
function W EIGHTED -S AMPLE(  , e) returns an event and a weight

L
x  an event with elements; u— 1
for  = 1 to do L
if ‹ ’ has a value ¼ ’ in e
then u—Cu SO
£ X J LNM
‹o’ M ¼
’9Á Ì4† p „ X o
‹ ’ [ [
else ¼ ’  a random sample from P X ‹ ’ Á
Ì † p KJ LM „X ‹ ’ [[
return x, u

Figure 14.19
34 Chapter 14. Probabilistic Reasoning

F C £
function MCMC-A SK( , e,  , ) returns an estimate of X ‹ Á e [
F F
local variables: N ` a , a vector of counts over , initially zero
Z, the nonevidence variables in 
x, the current state of the network, initially copied from e

initialize x with random values for the variables in Z

R
for ¶ = 1 to do
F
N ` k"a N ` k"a\7z where k is the value of in x
T
for each ’ in Z do
T T U T V
sample the value of ’ in x from P X ’ Á ÑX ’ [[ given the values of âµ X T ’ [ in x
return N ORMALIZE(N ` a ) F
Figure 14.21
 PROBABILISTIC
15 REASONING OVER TIME

function F ORWARD -BACKWARD(ev,

 ) returns a vector of probability distributions
M
inputs: ev, a vector of evidence values for steps z::
W

 , the prior distribution on the initial state, P X X [
X M
local variables: fv, a vector of forward messages for steps ::
b, a representation of the backward message, initially all 1s
M
sv, a vector of smoothed estimates for steps ze::

X
fv ` a T

Q
for M z to do M
Q Q
fv ` a F ORWARD X fv ` ™_za3: ev ` a0[Q
for M Q M
Q O
downto 1 do
Q
sv ` a N ORMALIZE X fv ` a b [
b  BACKWARD X b : ev ` a0[ Q
return sv

Figure 15.5

35
36 Chapter 15. Probabilistic Reasoning over Time

Y
function F IXED -L AG -S MOOTHING( p , ,6F6 , = ) returns a distribution over X YZA[
Y
inputs: p , the current evidence for time step
,6F6 , a hidden Markov model with ±
MO
± transition matrix T
= , the length of the lag for smoothing
static: , the current time, initially 1
\Y ^] Y
f, a probability distribution, the forward message P X ‹ Á pŒ [ , initially P RIOR ` ,6F6oa
_
B, the -step backward transformation matrix, initially the identity matrix
p YZA[2] Y
, double-ended list of evidence from ™ M _ M
to , initially empty
local variables: O Y A[ Y
: O , diagonal matrices containing the sensor model information

add p Y to the end of p Y A[2] Y

OY  diagonal matrix containing P X pYÁ ‹\Y [
if M b _ then
f  F ORWARD X f : pY [
remove p Y A[ ]
A Πfrom the beginning of p Y A[2] Y
OY A[  diagonal matrix containing P X p Y A[ Á ‹ Y A[ [
A Œ ?A Œ
?
B  OY A[ T BTOY
else B BTOY


D ?\{z
if M b _ then return N ORMALIZE X f O B1 [ else return null

Figure 15.8

C
function PARTICLE -F ILTERING(e, , = ) returns a set of samples for the next time step

C
inputs: e, the new incoming evidence
, the number of samples to be maintained
WC W
= , a DBN with prior P X X [ , transition model P X X ŒeÁ X [ , and sensor model P X E ŒÁ X Œ [
static: & , a vector of samples of size , initially generated from P X X [
C W
local variables: `
, a vector of weights of size

for  = 1 to do C
W
& [  ]  sample from P X X Œ Á X M &y` a0[
` Q
[  ]  P X e Á X Œ M ±L` a$[
C
&  W EIGHTED -S AMPLE -W ITH -R EPLACEMENT( , & , ` )
return &

Figure 15.18
16 MAKING SIMPLE
DECISIONS

function I NFORMATION -G ATHERING -AGENT( 
 


) returns an  
static: # , a decision network

integrate 


into #
® Ä 'X b [™
Ê  X'b “ [
8
¶  8 that maximizes a “
Ä X'value
® the b ' b
if a “ Ð
[ b Ê 
 
 
X “ [
then return R EQUEST( b “ )
else return the best action from #

Figure 16.9

37
17 MAKING COMPLEX
DECISIONS

c
function VALUE -I TERATION( 6o= , ) returns a utility function
l
inputs: 6o= , an MDP with states & , transition model , reward function * , discount d
c
, the maximum error allowed in the utility of any state
local variables: Í , Í | , vectors of utilities for states in & , initially zero
e
, the maximum change in the utility of any state in an iteration

Í |;e 
repeat
Íï 0
for each state  in & do
Ív| `  a *x` a¢\ d SVUW j X3„e:†
:„|$[¢Í ` i„ |Ja
fhg
Í | aF™ Í „ a Á b e then‚ i e  Á Í | ` ax™ Í ` „ a Á
until
elifk Á c XI`zQ™ d [º d `
return Í

Figure 17.5

38
 39

function P OLICY-I TERATION( 6o= ) returns a policy

l
inputs: 6o= , an MDP with states & , transition model
Í Í |
local variables: , , vectors of utilities for states in & , initially zero
m
, a policy vector indexed by state, initially random

Íï P OLICY-E VALUATION(m

repeat
, Í , 6o= )
!1
," +1
="H9 true
for each state  in & do
if SVUW f H
j X3„e:†
:„|$[¢Í ` „|Jacb H
j X3„e: m ` „a:„|$[¢Í ` „ |Ja then
‚^i ‚i
^
m ` aÇoU n6peSVUW f H j X3„e: †
: „ | ¢
[ Í `„ |a
!1>," 1+
=H2 false ^‚ i
until !"
," 1+1
=H
®
return

Figure 17.9
18 LEARNING FROM
OBSERVATIONS

function D ECISION -T REE -L EARNING(
ke 6Ã


 ,    , =
/ !1 ) returns a decision tree
inputs:
k 6Ã

 , set of examples
  3 , set of attributes
=
/ !1 , default value for the goal predicate

if
k 6Ã4
 is empty then return =
/ !1

else if all
ke 6Ã

 have the same classification then return the classification
else if    is empty then return M AJORITY-VALUE(
k 6Ã

 )
else

 4 C HOOSE -ATTRIBUTE(    ,
k 6Ã

 )


< a new decision tree with root test 

6{ M AJORITY-VALUE(
k 6Ã4
 ’ )
for each value d ’ of 
 do

k 6Ã4
 ’  ‰ elements of
k 6Ã

 with 
 M œ ’ Š
 !
 

B D ECISION -T REE -L EARNING(
k 6Ã4
 ’ :   2™ý
 , 6 )
add a branch to 

with label d ’ and subtree !4 


return 


Figure 18.6

40
 41

function A DA B OOST(
k 6Ã


 , . , Ï ) returns a weighted-majority hypothesis
C
inputs:
k 6Ã

 , set of labelled examples X ¼]Œ : –"Œ [::X ¼ : – [ q Kq
. , a learning algorithm
Ï , the number of hypotheses in the ensemble
C
local variables: w, a vector of example weights, initially ziº KC
h, a vector of Ï hypotheses
z, a vector of Ï hypothesis weights

for 6 = 1 to Ï do
h ` 6˜ae . (
k 6Ã

 , w)

8 0
for ¶ = 1 to do C
if h ` 6˜aX ¼ “ [ÃM K – “ then
8E
å\ w ` a
C r
for ¶ = 1 to do

r r ts
if h ` 6˜aX ¼ “ [ M – “ then w ` a w ` a ­
º"XIzQ™{
 e[
vuxwyp
w N ORMALIZE(w)
z ` 6˜a ˜XIzL™{
[º4
 
return W EIGHTED -M AJORITY(h, z)

Figure 18.12

function D ECISION -L IST-L EARNING(
k 6Ã4
 ) returns a decision list, or / $J!1

if
k 6Ã4
 is empty then return the trivial decision list ­ C

 a test that matches a nonempty subset
k 6Ã

 of
ke 6Ã


Y
such that the members of
k 6Ã4
 are all positive or all negative
Y
if there is no such then return / $J!1

if the examples in
k 6Ã

 are positive then y
Y I

 else y ­ zC

return a decision list with initial test and outcome and remaining tests given by
D ECISION -L IST-L EARNING(
k 6Ã

Й
k 6Ã

 ) Y
Figure 18.17
19 KNOWLEDGE IN
LEARNING

function C URRENT-B EST-L EARNING(
k 6Ã4
 ) returns a hypothesis

} 
any hypothesis consistent with the first example in
ke 6Ã


for each remaining example in
k 6Ã

 do
}
if
is false positive for then
}  }
choose a specialization of consistent with
ke 6Ã



}
else if is false negative for then
}  }
choose a generalization of consistent with
ke 6Ã


if no consistent specialization/generalization can be found then fail
}
return

Figure 19.3

function V ERSION -S PACE -L EARNING(
k 6Ã4
 ) returns a version space

a
local variables: , the version space: the set of all hypotheses

a  the set of all hypotheses

for each example
in
k 6Ã

 do
a a
if is not empty then  V ERSION -S PACE -U PDATE( ,
) a
return a
a
function V ERSION -S PACE -U PDATE( ,
) returns an updated version space
a  ‰ ^ »{a| ^ is consistent with
Š

Figure 19.5

42
 43

n
function M INIMAL -C ONSISTENT-D ET( , ) ) returns a set of attributes
n
inputs: , a set of examples
) , a set of attributes, of size 
for "
  X :V:g do
for each subset )L’ of ) of size  do
n
if C ONSISTENT-D ET ?( ) ’ , ) then return ) ’
n
function C ONSISTENT-D ET ?( ) , ) returns a truth-value
inputs: ) , a set of attributes
n
, a set of examples
}
local variables: , a hash table
n
for each example
in do
}
if some example in has the same values as
for the attributes )
but a different classification then return / J

}
store the class of
in , indexed by the values for attributes ) of the example

return !4

Figure 19.11
44 Chapter 19. Knowledge in Learning

function F OIL(
k 6Ã


 ,  3+
 ) returns a set of Horn clauses
inputs:
k 6Ã

 , set of examples
 +1
 , a literal for the goal predicate
local variables:  !"
 , set of clauses, initially empty

while
ke 6Ã


 contains positive examples do
 !"
2 N EW-C LAUSE(
k 6Ã

 , 3 +1
 )
remove examples covered by ( !1
from
ke 6Ã


add ( !1
to  !"

return ( !1


function N EW-C LAUSE(
k 6Ã


 , 3 +1
 ) returns a Horn clause
local variables:  !"
, a clause with 3 +1
 as head and an empty body
 , a literal to be added to the clause

k 
>=
=
k 6Ã

 , a set of examples with values for new variables

k 

=
=
ke 6Ã

Bˆ
k 6Ã


while
k 
>=
=
k 6Ã

 contains negative examples do
e C HOOSE -L ITERAL(N EW-L ITERALS( ( !1
),
k 
>=
=
ke 6Ã

 )
append  to the body of ( !1


k 
>=
=
k 6Ã

< set of examples created by applying E XTEND -E XAMPLE
to each example in
k 
>=
=
k 6Ã4

return ( !1

function E XTEND -E XAMPLE(
k 6Ã


, J$ 
  ) returns
if
k 6Ã4
satisfies J$ 
 
then return the set of examples created by extending
k 6Ã

with
each possible constant value for each new variable in  $ 
 
else return the empty set

Figure 19.16
20 STATISTICAL LEARNING
METHODS

function P ERCEPTRON -L EARNING(
ke 6Ã


 , Y
 u' ) returns a perceptron hypothesis
inputs:
k 6Ã

 , a set of examples, each with input x M k Œ ::k ‘ and output %
Y
 u' , a perceptron with weights · “ : M r X L
 , and activation function +

repeat
for each
in
ke 6Ã

 do
$ H ‘ k
a
n  D% “~} W · “ “ `
`p F
a ¬
™ Y
+ I
X $
 ][
· “  · “ \ š O n Ož+ | XI$][€Ožk
a
“ `
until some stopping criterion is satisfied
return N EURAL -N ET-H YPOTHESIS( Y
 u' )

Figure 20.22

45
46 Chapter 20. Statistical Learning Methods

function BACK -P ROP -L EARNING(
ke 6Ã


 , Y
 u' ) returns a neural network
inputs:
k 6Ã

 , a set of examples, each with input vector x and output vector y
Y
 u' , a multilayer network with . layers, weights · “ ³ ’ , activation function +

repeat
for each
in
ke 6Ã

 do
r
for each node in the input layer do “ ~k “`
a
 $
for = 2 to Ï do
’  H “ · “ ³’ “
’ ~ +>XI$ ’ [
Q
’ Ç+ | IX 0 ’ [‚OÉX – ’ `
aF™C† ’ [
for each node in the output layer do
m
for  = Ï ™ z to 1 do
for each node in layer do
m r  m
“ ~+ | XI$ “ [ H
’ · “ ³’ {
’
for each node Q in layer  \ z do
· “ ³ ’  · “ ³ ’ \ š OɆ “ O
m
’
until some stopping criterion is satisfied
return N EURAL -N ET-H YPOTHESIS( Y
 u' )

Figure 20.27
 REINFORCEMENT
21 LEARNING

function PASSIVE -ADP-AGENT(



) returns an action
inputs: 

4 , a percept indicating the current state  | and reward signal  |
m
static: , a fixed policy
j
6o= , an MDP with model , rewards , discount ƒ d
Í , a table of utilities, initially empty
C :
C ::i
, a table of frequencies for state-action pairs, initially zero
, a table of frequencies for state-action-state triples, initially zero
 , , the previous state and action, initially null

if  | is new then do Í [ | ] D| ; * [ | ] D|

if  is not null then do
C :
[  , ] and C [  , , | ]
increment
for each such that C ::i ::i
[  , , ] is nonzero do
l   DC :  : i C
[ , , ]
ÍÐ VALUE -D ETERMINATION( , Í , 6o= ) m :
[ , , ] / [ , ]

m
if T ERMINAL ?[  | ] then  , w null else  , LC | , `  | a
return

Figure 21.3

47
48 Chapter 21. Reinforcement Learning

function PASSIVE -TD-AGENT(

4 ) returns an action

inputs: 

4 , a percept indicating the current state  | and reward signal  |
m
static: , a fixed policy
Í , a table of utilities, initially empty
C
‚ , a table of frequencies for states, initially zero
 , ,  , the previous state, action, and reward, initially null

if  | is new then Í [  | ] D |

if  is not null then do
increment ‚ [  ] C
ZC
Í [  ]  Í ` a­\ š X ‚ ` a$[XIå\ d
Í `  | aF™ Í ` a0[
if T ERMINAL ?[ | ] then  , ,  null else  , , D| , ` |(a , | m
return

Figure 21.6

function Q-L EARNING -AGENT(
 


) returns an action
inputs: 

4 , a percept indicating the current state  | and reward signal  |
C„ : 
static: , a table of action values index by state and action
, a table of frequencies for state-action pairs
 , ,  , the previous state, action, and reward, initially null

if  is not null then do

C :
[ , ]
„  `| 4:a¢\  š XZC :  ` :i ea$[XIÂ\ d SgUW f i „F` †
increment
„F` 4:a F | : | ax™ „F` 4:a0[
else  , , D | , Uon6e p SgUW f i R X „F` †null | : | a3:>C :  ` † | : | a0[ ,  |
if T ERMINAL ?[ ] then , ,

return

Figure 21.11
22 COMMUNICATION

function NAIVE -C OMMUNICATING -AGENT( 



) returns  
static: ¡¢- , a knowledge base
 3  
, the current state of the environment
  , the most recent action, initially none
   
2 U PDATE -S TATE(    
,   , 
 

)
u=< S PEECH -PART( 


)

6o 4 < D ISAMBIGUATE(P RAGMATICS(S EMANTICS(PARSE( u= ))))
if u= = C­Y
and e  is not a S AY then /* Describe the state */
return S AY(G ENERATE -D ESCRIPTION(    
))
else if T YPE[ 
6o 4  ] = Command then /* Obey the command */
return C ONTENTS[ 
6o 4  ]
else if T YPE[ 
6o 4  ] = Question then /* Answer the question */
4uy
8 A SK( ¡¢- , 
6o 4  )
return S AY(G ENERATE -D ESCRIPTION( 4uy
 ))
else if T YPE[ 
6o 4  ] = Statement then /* Believe the statement */
T ELL( ¡N- , C ONTENTS[ 
6o 4  ])
/* If we fall through to here, do a “regular” action */
return F IRST(P LANNER( ¡¢- ,  3  
))

Figure 22.3

49
50 Chapter 22. Communication

function C HART-PARSE( u= , +e 6F6o  ) returns ," 

," 

array[0. . . L ENGTH( u= )] of empty lists
X X
A DD -E DGE( ` : : ± | ! ± a) …
for "
  from 0 to L ENGTH( = ) do

u
S CANNER(  , uy = [  ])
return ,1 
procedure A DD -E DGE(
=+
)
/* Add edge to chart, and see if it extends or predicts another edge. */
if
=+1
not in ,"  [E ND(
=+
)] then
append
=+
to ,1  [E ND(
=+1
)]
if
=+
has nothing after the dot then E XTENDER(
=+1
)
else P REDICTOR(
=+1
)
procedure S CANNER(¶ , u= )

Q r
for each ` : : ) š! …
/* For each edge expecting a word of this category here, extend the edge. */
- › a in ,"  [¶ ] do

u 
 
 = is of category - then
if
Q r
A DD -E DGE( ` : \Fze:8Î šµ h!› a) …
procedure P REDICTOR( ` : ¶ : ) š -_› a ) ! …
/* Add to chart any rules for - that could help extend this edge */
for each X - ! d
[ in R EWRITES -F OR( - , +e 6F6o  ) do
A DD -E DGE( ` : : - r r a) ! … d
procedure E XTENDER( ` ¶ :'
: - 1a ) ! d …
/* See what edges can be extended by this edge */
†
p  the edge that is the input to this procedure
Q r ‡!
for each ` : :<Î š …
µ | › a in ,"  [¶ ] do
if µ M µ | then
A DD -E DGE( ` :
:ïÎ Q ;ˆ h! š;p › a) † …
Figure 22.9
23 PROBABILISTIC
LANGUAGE PROCESSING

®
function V ITERBI -S EGMENTATION( 
k , ) returns best words and their probabilities
inputs: 
k , a string of characters with spaces removed
®
, a unigram probability distribution over words
 L ENGTH( 
k )
u=< empty vector of length L \{z

 4 vector of length L \{z , initially all 0.0

 [0]  1.0
/* Fill in the vectors best, words via dynamic programming */
for  = 0 to do L
for ¶ = 0 to ™_z do Q
u=vD 
k [ : ]
u— L ENGTH( u= )
r Q
® O
if [ uy = ] 
 [  - u ]  
 [  ] then

 [ ]  ® [ u= ] ý
 [ ™
Q O
] Q Š‰
u= [ ] Cu=Q
/* Now recover the sequence of best words */
sequence  the empty list
Q  L
while b Q X
do
push u Q
 = [ ] onto front of 
5!4
>

 Q Q
™ L ENGTH( u= [ ]) Q
/* Return sequence of best words and overall probability of sequence */
return 
5!4
>
, 
 [ ] Q
Figure 23.2

51
24 PERCEPTION

function A LIGN( $6o +1
, 6o=
 ) returns a solution or failure

inputs: 06o +1
, a list of image feature points
6o=
 , a list of model feature points


for each Ì Œ :
Ì Ž :>Ì in T RIPLETS( $6o +
) do
U U
for each ¦Œ : ;Ž : U‚
in T RIPLETS( 6o=
 ) do
„  U
 F IND -T RANSFORM( Ì Œ :>Ì Ž :>Ì , ¦Œ : ;Ž : U U‚ )
„
if projection according to explains image then
return „
return failure

Figure 24.22

52
25 ROBOTICS

C
function M ONTE -C ARLO -L OCALIZATION( , ° , , 6o=
 , 6o  ) returns a set of samples
inputs: , the previous robot motion command
z, a range scan with Ï readings ° Œ ::° ;‹
C
, the number of samples to be maintained
6o=
 , a probabilistic environment model with pose prior P X X [ ,
£ W ^Œ Œ
W W
motion model P X X ŒÁ X : Î [ , and range sensor noise model X Á L[
6o  , a 2D map of the environment
>
C
static: & , a vector of samples of size , initially generated from P X X [ W
local variables: `
, a vector of weights of size C
for  = 1 to do C
& [  ]  sample from P X X ŒeÁ X M &y` a3:<Î W W M 1[
` [ ]  1
for ¶ =1 to Ï do
°y E XACT-R ANGE(¶ , & [  ], 6o  )

` ` 3a `•` aŽs £ X^Œ M ° “ ;Á Œ  M °[
 C
&  W EIGHTED -S AMPLE -W ITH -R EPLACEMENT( , & , ` )
return &

Figure 25.8

53
26 PHILOSOPHICAL
FOUNDATIONS

54
27 AI: PRESENT AND
FUTURE

55