Soccer like ball control using a 3-dimensional biped

simulation
Aashith Kamath Manjeshwar, Natia Chachkhiani, Justin Morgan
Computer Science
University of California, Los Angeles
California 90024
Email: aashithk@cs.ucla.edu - natia@cs.ucla.edu - jjmorgan@ucla.edu

AbstractIn this project we implement a biped with soccer

like ball control abilities. We make use of the three dimentional
physics-based biped locomotion framework called Simple Biped
Locomotion Control (SIMBICON) and expand its control framework to enable the biped to perform this new behavior. We define
this behavior in terms of two primary objectives. One involves
asking the biped to dribble the ball towards the goal as in normal
soccer play. The second object is to kick the ball from a distance
towards the goal post. We define new controllers to enable the
model to walk towards the moving ball and control it for the
first objective. We also implement the controller to kick the ball
from a distance towards a stationary goal post. We modify the
balance feedback function for both these controllers to ensure
that the biped is robust while executing these objectives. We also
make use of feedback error learning to keep the biped stable
while transitioning between the different actions and to enable
it to anticipate future torques.
Index TermsBiped, FSM, ball, soccer.

I. I NTRODUCTION
Biped locomotion has been one of the most challenging
locomotion tasks over the years. The interest in it has been
growing since the advancements of humanoid robots. Biped
locomotion is especially challenging because it involves highdimensional states and actions and they are very unstable.
Of the number of algorithms that have been proposed over
the years, one of the most successful models on physics
based simulations involves developing controllers to drive
forward dynamics simulations. By using this approach, many
researchers have been successful in creating robust biped
walking frameworks that exhibit natural motion using and
can resist external turbulences. Our goal in this project is
to enable the biped to perform soccer like ball control. We
define this with two main objectives. The first is to enable
the biped to dribble the ball towards a target successfully
without falling over or losing control of the ball. The
second objective is to enable the biped to perform a longer
distance kick as is commonly seen in soccer. We also wish
to ensure that the stability of the biped is maintained during
this process. To achieve these objective, we extend the
SIMBICON (SIMple BIped locomotion COntrol) framework.
This framework involves a finite state machine with all
individual joints attempting to drive towards their target
angles using proportional derivative controllers. We achieve
the first and second objective by defining a new controller

and a new state/trajectory for each task. For each controller,

we also modify the balance feedback function to ensure that
the biped is aware of the ball and its contact forces. Also
we use feedback error learning to achieve low gains control
in the model and enable the biped to anticipate the future
torque and thus perform the task in a smoother manner. We
also enable the biped to walk towards a moving target, in
our case, the ball . When it is near the ball, we enable the
biped to control/dribble the ball as in a regular soccer play.
This model also ensures that the biped is stable during both
operations and is robust to different types of contact with the
ball.
II. R ELATED W ORK
Literature related to biped locomotion is exhaustive. Here
we describe a few of them.
One of the approaches used is Reinforcement learning,
which is applied in different ways to learn to control the
walking and the movement. In [4] the biped learns to
appropriately place the swing leg using RL. The model
transitions from a state at the middle of a step and foot
location to a state at next middle of a step. Unfortunately,
this model is not yet sufficiently advance to handle a task like
ball control. However, a genetic algorithm to evolve this task
with certain initial motion capture information could lead to
a successful approach.
In later years, algorithms based on central pattern
generators (CPG) became more and more popular. Authors
in [2] discuss a CPG based approach which leverages phase
oscillators for bipedal locomotion where the designer with
a little prior knowledge can incrementally add basic motion
primitives, accomplishing bipedal walking as a result.
An interesting approach in [3] uses crossing continuous
time recurrent neural network to simulate CPG in order to
improve the ability to produce rhythmic activation behaviors
in neural net. RL is also used in conjunction with a CPG
[4], which is inspired by animals control mechanism. The
learning algorithm in this approach is based on policy gradient
method.
Another interesting work done using physically simulated
biped is described in [5]. This work discusses an optimization
method which transforms motion captured or kinematically

produced motion into physically realistic, balance preserving

simulated motion. For our purposes we chose the SIMBICON
model, which produces stable locomotion behaviors using
forward dynamic simulation. This model has many similar
components with [5]. The source code for a very basic
implementation of this framework is open for future research
and available for others to extend. This makes it a great
starting point for the project. In the next section, we give
brief overview of the base model we use.
III. SIMBICON OVERVIEW
To achieve our goal we make use of the biped framework
called SIMBICON (SIMple BIped CONtrol). SIMBICON is
based on a finite state machine (FSM) or pose control graph.
The Figure 1 shows the finite state machine for walking in
a 2D biped. It makes use of balance feedback as well as
the torso orientation with respect to the world axis to ensure
balance control for the biped. In order to create more natural
and robust movements, the feedback error learning is used in
the model to learn the feed forward torques and anticipate
future torques. The controller can be designed manually by
tuning a number of states with specific parameters for each
state. We can also design the controller by making use of
motion capture data which makes its design more precise and
easy.

does not ensure any balance control and also does not yield
natural locomotion. To ensure balance control, a small set of
modifications are applied to the model. The first one is that
both torso and the swing-hip should have target angles that are
calculated with respect to the real world while the remaining
target joint angles are expressed relative to its parent joint.
This is accomplished by using a virtual PD controller that
operates in the world frame to compute a net torso torque
torso and a virtual PD controller that operates in the world
frame to compute swing hip torque B . These virtual torques
torso and B are realized using only the internal stance hip
torque. Figure 2 shows us the elements of the balance control
strategy. The second modification introduced is the addition of
a balance feedback term which continuously changes swing
hip target angle as a linear function of the center of mass
position and velocity. The feedback law used is expressed as
follows
d = d0 + cd d + cv Where
d is the target angle used for PD control
d0 is the default fixed target angle as described in the FSM
d is the horizontal distance from the stance ankle to the
center of mass
v is the velocity of the center of mass
cd feedback gain parameter
When extending this balance feedback to the 3D model,
this strategy is applied to both sagittal and coronal planes.
The above FSM model is resistant to external disturbances,
but its movements are still pretty stiff because of the high
gains used in the feedback loop. In order to create a more
natural locomotion, feedback error learning (FEL) is applied
to the biped. FEL allows the learning of inverse dynamics of
the system to produce the given motion trajectories without
actually calculating them. Phase is uniformly divided in N
(20 - 1000) bins. Each bin makes use of a filter represented
as:
vf0 f = (1 )vf f + (vf f + vf b ) Where
Learning rate is 0.1.
vf f &vf b represent the feed forward & feedback torques.

Fig. 1. Finite State Machine

The movement of the biped is achieved by transitioning

from state to state. Each state represents a target pose composed of the joint angles with respect to their parent joints.
The transition between the states is done by using proportional
derivative (PD) controllers computed using = kp (d )kd .
Transition between the states is done either when foot touches
the ground or after a certain time has elapsed. Just this however

Thus, we learn the feed-forward torques as a function of the

current phase of the motion. FEL may be applied to individual
joints or to all joints concurrently. High learning rates might
lead to convergence problems and this must generally be
avoided. In the next section, we look at how this model was
extended to achieve the objectives.
IV. E XTENDING SIMBICON FOR BALL CONTROL AND
K ICK MOTION
Our goal here is to try and create a three dimensional biped
simulation that can perform Soccer like ball control motions.
We break this task into two independent objectives. The first
objective is to enable the biped to kick a ball starting from

Fig. 2. Components of the balance control strategy

a standing position. In this action, the ball release at a point

in the world and the biped must walk towards it and kick
the ball a certain distance. While implementing this project,
we further extend this objective by creating the additional
constraint of kicking the ball towards a predefined goal as in
the case of a penalty kick. The second objective is to enable
the biped to walk towards the ball and control the ball as
in regular Soccer play. We also extend this in the project to
control the ball towards a goal post. We explain the details of
implementing these two objectives in the next section. Finally
we also want to ensure that the model displays robustness
during the above tasks and to direction changes using balance
feedback and feedback learning.
A. Soccer Ball control Implementation technique
To implement the soccer ball control, we follow the steps
that we as humans would perform for the same. As the first
step, we develop a new control strategy that is layered on
top of the original strategy for simple biped locomotion. The
biped is first given information about the proximity of the ball
in real time. Thus, it is capable of knowing the location of
the ball. Next, to find a path to the ball, we modify the target
state by introducing deviations in the orientation of the biped
using the real time position of the ball. We now derive the
desired biped trajectory as a function of the current position
of the biped and the next modified state. This process ensures
that the biped is able to find and walk towards the ball even
if the ball is in motion.
Once at the ball, we define the next goal of the biped as
ensuring the ball is within a short distance of its center of mass
while it loco-motes to a given target. In our implementation,
this is the goal post. We do this by making use of the balance
function, a modified trajectory for ball tap and the feedback
error learning module. The balance function is updated to

take the ball contact into its calculations. This ensures that
the biped is aware of each contact and can alter the next
state to ensure balance. Because the low-gain motions used
here are less robust than the high-gain motions, we increase
the gains for a short duration after impact. This is applied
in both the sagittal and coronal planes. This helps the biped
stay stable on both axes of motion after each tap of the ball.
Next we define the ball tap trajectory which is activated
when the foot of the biped is very close to the ball. In this
trajectory we slightly increase the speed of the swing leg and
also modify its trajectory to try and kick the ball such that
it does not deviate from the direction of the goal post. This
helps the biped ensure that the ball is always in front of it
and also to drive the ball towards the target. Next we modify
the feedback error learning module to consider knowledge of
proximity of the ball, mass of the ball and contact force. This
helps learn the feed-forward torques in each phase during
the ball control and also makes the ball control more stable
since the biped can anticipate the torques in each phase of
the control. We include a link at the end of the report that
contains a video of this process.
B. Kick Implementation technique
To implement the Kick action similar to that of a penalty
kick, we again make use of certain techniques that humans
would use for the same. We develop a new control strategy
that is layered on top of the original strategy for simple biped
locomotion. Here again, the biped is first given information
about the proximity of the ball in real time and knows the
location of the ball. We use the strategy mentioned in the ball
control implementation to find a path to the ball. We modify
the target state by introducing deviations in the orientation of
the biped using the real time position of the ball and use this
to derive the desired biped trajectory. Once the biped is close
to the ball, its goal now is defined as kicking the ball towards a
target. In our implementation this is the goal post. We do this
primarily by modifying the balance function and defining a
new target state for ball kick. The balance function is updated
to take the ball contact into its calculations. This ensures that
the biped is aware of the contact event and can alter the next
state to ensure balance after the kick. This is again applied in
both the sagittal and coronal planes.
Next we define the ball kick state which is activated when the
foot of the biped is very close to the ball. In this state, the
swing leg is raised much higher to similar a kick. We learn the
modified kick state for this action by studying motion capture
data. The transition to this state ends when the foot comes
in contact with the ball and regular locomotion is resumes.
We also modify the trajectory between the previous normal
locomotion state to this state to simulate the steps that humans
take to kick the ball for a longer distance. We first increase the
speed of the swing leg during the trajectory as it nears the ball.
Just before the toe touches the ball, we freeze the swing legs
joints until ball contact is completed to simulate the tightening
of the muscles and ensure that the joints are more rigid for

the period of the kick. This also ensure that more force is
transferred to the ball during the kick. We also modify the
trajectory of the kick to try to kick the ball towards the goal.
This is done by defining a function that take the position of the
goal and the current position of the ball to return the segment
of the ball that the foot should ideally touch. For instance,
if the ball is on the right side of the goal post, then function
would return the right segment of the ball. Kicking the ball on
the right segment ensures that it travels towards the left and
thus towards the goal. Once the kick is executed the biped
reverts back to the normal locomotion state sequence.
V. I MPLEMENTATION AND R ESULTS
The project was implemented using the C++ based Open
dynamics engine as the physics SDK. Open dynamics engine
is an open source and high performance library for simulating
rigid body dynamics. Since the platform is mature and has
a C++ API for low level control of the model, we can
define advanced joint types and also define integrated collision
detection with friction. We extend the simple biped locomotion
implementation of the SIMBICON model to perform advanced
operations such as ball control. We make use of OpenGL
(Open Graphics Library) for the vector graphics implementation. OpenGL is a cross-language, multi-platform application
programming interface which is used for rendering 2D and 3D
vector graphics. The API is used to interact with a graphics
processing unit, to achieve hardware-accelerated rendering.
The parameters of the world are as in the real world. The
gravity is chosen as 9.8. The biped in the simulation weighs a
total of 90 kilograms. The biped has a 70 kg trunk including
the arms, 5 kg upper legs, 4 kg lower legs, and 1 kg feet.
The model has human-like proportions and mass distributions.
A spring-and-damper penalty-force ground contact model is
applied to points at the front and back of the feet. Proportional
derivative controller gain values of kp and kd are set to 300
Nm/rad and 30 Nms/rad respectively for all joints J as per the
original SIMBICON project. For the basic locomotion, the
walking gaits are modeled by using 4 states in the finite state
machine. Transitions between states occur after fixed durations
of time, or, for other states, after a new foot contact has been
established. The Degrees of Freedom (DOF) of the model is
shown in figure 3. For the 3D model, there are 34 degrees of
freedom in total. 28 of which are internal degrees of freedom.
The ball weight in the implementation is 15 ounces and
the friction and restitution coefficients are chosen to mimic
the behavior of a real football. In one of the demo videos
we also see the ball spinning after touching the ground after
the kick. This is similar to that of a football that spins when
kicked on one side. We choose the simulation time step for
operation as 1/2000th of a second. For the Balance Feedback
Gain Parameters, cd , cv we use cd = 0.5 and cv = 0.2 for
the swing hip in all states, in both the coronal and sagittal
planes. The model also includes the same robustness that
was observed in the SIMBICON model and makes use of
the balance controller. This ensures that the biped can deal

Fig. 3. Degrees of Freedom of the 3D Biped

with unexpected environmental disturbances automatically.

The SIMBICON project suggests that future work extending
the model will need to develop controllers constructed,
either manually or using motion capture data. We define the
controller for the ball control task manually since motion
capture for this dynamic task is not very useful. However, we
study the motion capture information to define the kick state
that we use for the kick implementation. The entire project is
developed in the Visual studio IDE.
We include a video demo of both, the ball control and the long range kick at the web link :
http://bipedsoccer.weebly.com/ . We also include a few frames
from those videos below of each demo video.
A. Target pose (State) definition for the kick implementation
We used a duplicate biped to display the target pose at each
instant in the bipeds locomotion. Using this, we show in figure
4 and figure 5, the target state we defined by studying a kicks
motion capture information. This state is activated as the next
state when the ball is in kicking proximity. Notice that the
target pose of the swing leg for the next step is much higher
that a normal walking gait. At the beginning, the left leg has
just touched the ground ending the previous state and the right
leg is now about to start its swing and is the swing leg. Thus
the target pose showing duplicate has been updated with the
target state of the right leg.

Fig. 4. Kick state activation at the start of the step

Fig. 7. Force Vector when both legs in contact with the ground

Fig. 8. Force vectors when a single leg is in contact with the ground
Fig. 5. Kick state continues until complete or contact with ball is achieved

C. Soccer Ball control

B. Soccer world
The world of the biped is shown in figure 6 and has been
defined to include a goal post that it must target for both
the tasks. A ball is dynamically introduced into the world by
using an interactive controller at the top left of the screen. The
surface of the field has been defined with 0.4 as the friction
co-efficient.
The vector of contact forces is shown in figures 7 and 8. It is
indicated by using arrows along the relevant axis. In the first
figure, we see the contact forces with both feet on the ground
at the end of a step. In the second figure we see the contact
forces of only one leg just before the step is completed.
Another point to note in the images is that the goal post
has been drawn as a rectangular object in the implementation
colored in dark green for easy identification. This is the target
that the biped is given when performing both, the ball control
as well as the kick.

We include the pictures of a video demo of the soccer ball

control in the figures from 9 to 18. In the soccer ball control
demo, we release the ball at short distance from the biped. In
the first two figures, we see the biped approaching the ball
using the modified set of states as mentioned above. Once it
reaches the ball, it follows the ball control technique. We see
this in the remaining 7 images.

Fig. 9. Soccer Ball control: Approach

D. Kick Implementation

Fig. 6. Soccer world

We include the pictures of a video demo for soccer ball

kick from figure 19 to 27. In the kick implementation demo,
we release the ball at short distance from the biped again. To
better display this, we release the ball at an angle and observe
the biped walking toward the location of the ball. In the first
three figures, we see the biped approaching the ball using the
modified set of states as mentioned above. Once it reaches the

Fig. 10. Soccer Ball control: Approach

Fig. 14. Soccer Ball control

Fig. 11. Soccer Ball control: Initiation

Fig. 15. Soccer Ball control

Fig. 12. Soccer Ball control

Fig. 16. Soccer Ball control

Fig. 13. Soccer Ball control

Fig. 17. Soccer Ball control: Goal !!

ball, its next state is the modified kick state. In addition to this
target state, the trajectory to the state is also modified with a
much higher speed of approach towards the ball of the swing

leg. Also when the swing leg is very close to the ball, the joints
are frozen to prevent the knee from collapsing and to simulate
the muscles tightening as is the case in a natural kick. We also

Fig. 18. Soccer Ball control: Goal !!

Fig. 21. Kick: Approach

see that the kick is on the center of the ball since the goal post
is on a straight line from the current position of the ball. This
ensures that ball does travel towards the predetermined target
of the goal post. Another interesting phenomenon we notice in
the videos is that the ball sometimes tends to spin slightly after
some distance when kicked with more force. This is similar to
the longer distance real ball kicks we see during actual soccer
play.

Fig. 22. Kick: Activation of the kick state

Fig. 19. Kick: Approach

Fig. 23. Kick: Speed of the trajectory is increased to increase force of contact
and joints of the swing leg are made rigid

Fig. 20. Kick: Approach

VI. L IMITATIONS
One major limitation of this model is that the task of
producing controllers from motion capture data is not fully
automated and we still need to manually tune the required
feedback gain constants. We however, believe that a learning

Fig. 24. Kick: Post-kick balance steps

algorithm can be used here to arrive an optimized value.

Also, another disadvantage is that the current gaits are not

Fig. 25. Kick: Post-kick balance steps

a dynamic model, we will be able to make use of robotic

bipeds for complicated tasks like completely robotic game
playing. The model used here leverages the finite state machine
framework to achieve one such task and we hope to leverage
this to perform more complex tasks in the future. We have
shown how to develop a controller for walking towards a
ball, controlling it towards a target and also for kicking the
ball. By modifying the balance function, we also enable the
biped to maintain stability during these actions and to display
robustness. There are a large number of directions that can
be pursued. This stability means that we may also apply this
model to humanoid robots. We hope to extend this model in
future work to create an entire team of players which might
then play against another such team. This would lead to highly
realistic game play. In addition, with the advent of virtual
reality wearables, integrating real physics into virtual reality
is gaining ground in the gaming industry and feel that such
models can be leveraged for the same.
ACKNOWLEDGMENT

Fig. 26. Kick: Goal!!

We would like to specially credit Stelian Coros, Philippe

Beaudoin and Michiel van de Panne from the University of
British Columbia for open sourcing the SIMBICON framework code which was extremely useful for this project and
acted as a base for our implementation.
R EFERENCES

Fig. 27. Kick: Goal!!

optimized for energy efficiency and this applies to the soccer

ball control and the kick motion as well. We also do not
account for any reaction delay that is displayed by humans.
This helps the model react almost instantly to any impact
and gives it an unnatural stability. Finally, we also notice that
with a simulation time step of 1/2000th of a second, there is
a lot of processing performed for each step in the trajectory
between two states. This leads to the render slowing down to
a lower frame rate of around 8-15 per second. However, this
issue may be circumvented with more processing power.
VII. C ONCLUSION & F UTURE WORK
Any kind of locomotion with a biped has its set of challenges and the existing models need to evolve until they arrive
at a solution that can perform tasks as diverse as humans.
By making bipeds more robust to reactions and defining such

[1] KangKang Yin , Kevin Loken , Michiel van de Panne SIMBICON: simple
biped locomotion control, ACM Transactions on Graphics (TOG), v.26
n.3, July 2007.
[2] Vitor Matos , Cristina P. Santos. Towards goal-directed biped locomotion:
Combining CPGs and motion primitives .
[3] J. Santos and A. Campo Biped locomotion control with evolved adaptive
center-crossing continuous time recurrent neural networks, Neurocomputing, vol. 86, pp. 86-96, 2012.
[4] T. Mori, Y. Nakamura, M. Sato, and S. Ishii., Reinforcement learning
for a cpg-driven biped robot., Proceedings of the Nineteenth National
Conference on Artificial Intelligence, AAAI, 2004.
[5] SOK, K. W., KIM, M., AND LEE, J. , Simulating biped behaviors from
human motion data., ACM Trans. on Graphics (Proc. ACM SIGGRAPH)
2007
[6] http://bipedsoccer.weebly.com Link to project resources and video demo,