The Lanczos

Algorithm for
Large-Scale
Eigenvalue
Problems

Sam Handler

The
The Lanczos Algorithm for Large-Scale
Eigenvalue
Problem Eigenvalue Problems
The Need for
Eigenvalue
Solvers

The Lanczos Sam Handler

Algorithm

The
Orthogonality
Problem
08/01/07
Implementation

Future
Directions

Questions
 The Eigenvalue Problem

The Lanczos
Algorithm for
Large-Scale
Eigenvalue
Problems

Sam Handler Given a square matrix A, find values λi and vectors vi

such that
The
Eigenvalue
Problem
Avi = λi vi
The Need for
Eigenvalue
Solvers

The Lanczos
Algorithm Values λi are called eigenvalues
The
Orthogonality
Problem

Implementation Vectors vi are called eigenvectors

Future
Directions

Questions
 Quantum Mechanics

The Lanczos
Algorithm for
Large-Scale
Eigenvalue
Problems The Time-Independent Schrödinger equation
Sam Handler

The HΨ = E Ψ
Eigenvalue
Problem
where H is the Hamiltonian operator, Ψ is a wave
The Need for
Eigenvalue function, and E is the energy in the system.
Solvers

The Lanczos
Algorithm This can be simplified so that H can be represented as a
The
Orthogonality
matrix and Ψ as a vector.
Problem

Implementation Eigenvalues correspond to energy levels of the system; the

Future
Directions
eigenvectors represent the corresponding wave-functions.
Questions
 Graph Theory

The Lanczos
Algorithm for
Large-Scale
Eigenvalue
Problems Given a graph G , the centrality of a vertex is a measure of
Sam Handler how “important” a vertex is to a graph.
The
Eigenvalue
Problem The importance of a node is proportional to the sum of
The Need for the importance of the nodes adjacent to it.
Eigenvalue
Solvers

The Lanczos
Algorithm Centrality is found by determining the eigenvector
The associated with the largest eigenvalue.
Orthogonality
Problem

Implementation This measure forms the basis for Google’s PageRank—

Future
Directions
algorithm.
Questions
 The Lanczos Algorithm

The Lanczos
Algorithm for
Large-Scale
Eigenvalue
Problems

Sam Handler

The
Eigenvalue
Reduces a large, complicated eigenvalue problem into a
Problem smaller, simpler one
The Need for
Eigenvalue
Solvers Approximates the eigenvalues of a matrix
The Lanczos
Algorithm

The
Finds the smallest and largest eigenvalues fastest
Orthogonality
Problem

Implementation

Future
Directions

Questions
The Lanczos A is the matrix, q1 is a random vector with |q1 | = 1
Algorithm for
Large-Scale
Eigenvalue
Problems
q0 = 0, β1 = 0
Sam Handler
for i = 1 to m:
u = Aqi − βi qi−1
The
Eigenvalue αi = u · qi
Problem
u = u − αi qi
The Need for
Eigenvalue βi+1 = |u|
Solvers
qi+1 = u/βi+1
The Lanczos
Algorithm

The Then
find the eigenvalues of 
Orthogonality
Problem
α1 β2
 β2 α2 β3 
Implementation  
. ..
β3 . .
 
Future
Directions
T = . 

 . .. α 
m−1 βm
Questions  
βm αm
 The Orthogonality Problem

The Lanczos
Algorithm for
Large-Scale
Eigenvalue
Problems

Sam Handler

The
Eigenvalue
Problem Each qi should be orthogonal to all other q vectors.
The Need for
Eigenvalue
Solvers Due to limited precision when storing vectors, new q
The Lanczos vectors slowly become less orthogonal.
Algorithm

The
Orthogonality
Problem

Implementation

Future
Directions

Questions
 Reorthogonalization

The Lanczos
Algorithm for
Large-Scale
Eigenvalue
Problems

Sam Handler Periodically reorthogonalize the current q vector against

all previous q vectors.
The
Eigenvalue
Problem
Takes a lot of time - is only done when necessary
The Need for
Eigenvalue
Solvers
Use simple recurrence relations to estimate level of
The Lanczos
Algorithm nonorthogonality - reorthogonalize when this level gets too
The large.
Orthogonality
Problem

Implementation In practice, reorthogonalize about every 10-15 iterations.

Future
Directions

Questions
 Implementation Notes

The Lanczos
Algorithm for
Large-Scale
Eigenvalue
Problems Test the eigenvalues every 10 loops (adjustable).
Sam Handler

The Test for convergence by testing the sum of the smallest (or
Eigenvalue
Problem largest) eigenvalues.
The Need for
Eigenvalue
Solvers The eigenvectors of A can be calculated as
The Lanczos  
Algorithm

The
Orthogonality
vi =  q0 . . . qn  wi
Problem

Implementation

Future where wi is an eigenvector of T .

Directions

Questions
 Performance

The Lanczos
Algorithm for
Large-Scale
Eigenvalue
Problems Performance is largely determined by disk speed.
Sam Handler

The Example: On a matrix of size n = 108,384, performing

Eigenvalue
Problem 1000 iterations took 4437 seconds, but only 269 seconds
The Need for (6%) were spent performing computations; the rest were
Eigenvalue
Solvers spent waiting for the disk.
The Lanczos
Algorithm

The
Orthogonality
The time spent waiting for disk should decrease with
Problem larger vectors.
Implementation
For a given operation, computing time increases linearly
Future
Directions
with vector size, while load time is nearly constant.
Questions
 Future Directions

The Lanczos
Algorithm for
Large-Scale
Eigenvalue
Problems
Goal: Further increase speed and scale of calculations
Sam Handler

The
Eigenvalue
Better handling of vector storage
Problem
Keep vectors in memory longer
The Need for
Eigenvalue
Solvers

The Lanczos Store fewer vectors

Algorithm

The
Faster to regenerate vectors than load them
Orthogonality
Problem

Implementation Parallelize for multiple-processor machines

Future
Directions

Questions
The Lanczos
Algorithm for
Large-Scale
Eigenvalue
Problems

Sam Handler

The
Eigenvalue
Problem

The Need for Questions?

Eigenvalue
Solvers

The Lanczos
Algorithm

The
Orthogonality
Problem

Implementation

Future
Directions

Questions