CSE291D Lecture 18

Social network models

(continued)

1
 Announcements
• HW2 is back, at the front of the room, as well as solutions

• Please turn in HW4 at the front

• HW5 is out on Piazza, hard copy at the front

– The worst homework will drop, so HW5 is effectively optional,
if you want to focus on project/exam.

– Due 06/09, midnight (lectures will be over then).

– Submit to my office (under the door if I’m not there) or by email

to me and the TA.

2
 Evaluations

• Please be sure to submit evaluations for both your

instructor and TA, if you have not done so already.

• This will help us a lot!

• (Thanks, if you have already done this)

• I understand that you have been emailed a link to do

this.

3
Exponential family random graphs
(ERGMs)

Arbitrary sufficient statistics

Covariates (gender, age, …)

E.g. “how many males are friends with females”

4
 Exponential family random graphs
(ERGMs)
• Pros:
– Powerful, flexible representation
– Can encode complex theories, and do substantive
social science
– Handles covariates
– Mature software tools available,
e.g. ergm package for statnet

5
 Exponential family random graphs
(ERGMs)
• Cons:
– Usual caveats of undirected models apply
• Computationally intensive, especially learning
• Inference may be intractable, due to partition function

– Model degeneracy can easily happen

• “a seemingly reasonable model can actually be such a
bad mis-specification for an observed dataset as to
render the observed data virtually impossible”
– Goodreau (2007)

6
 Triadic closure

If two people have a friend in common, then there is an increased likelihood

that they will become friends themselves at some point in the future.

7
 Measuring triadic closure
• Mean clustering co-efficient:

+
8
Simple ERGM for triadic closure
leads to model degeneracy

Depending on parameters, we could get:

• Graph is empty with probability close to 1

• Graph is full with probability close to 1

• Density, clustering distribution is bimodal, with little

mass on desired density and triad closure

MLE may not exist! 9

 Handcock, M. S., Hunter, D. R., Butts, C. T., Goodreau, S. M., & Morris, M. (2008). statnet: Software tools for the
10
representation, visualization, analysis and simulation of network data. Journal of statistical software, 24(1), 1548.
 What is the problem?

Completes two triangles!

If an edge completes more triangles, it becomes overwhelming likely to exist.

This propagates to create more triangles …

11
 Solution
• Change the model so that there are diminishing
returns for completing more triangles
– A different natural parameter for each possible number of
triangles completed by one edge
– Natural parameters parameterized by a lower-
dimensional , e.g. encoding geometrically decreasing
weights (curved exponential family)

• Moral of the story: ERGMS are powerful, but

require care and expertise to perform well
12
 Latent variable models
for social networks
• Associate each node (actor) i with a latent
representation zi

• Make the probability of an edge between

i and j depend on their latent representations

inverse link function controls density

interaction between latent variables covariates 13
 Inference algorithms
• The inference and learning methods we have already
studied in this course are typically used

– EM

– Variational Bayes

– Variational EM

– MCMC
• Gibbs, collapsed Gibbs, Metropolis-Hastings

14
 Latent Representations
Cycling Fishing Running Tango Salsa Waltz
Alice 1
• Latent class Bob 1
Claire 1

• Mixed Cycling Fishing Running Tango Salsa Waltz

Alice 0.2 0.4 0.4
membership
Bob 0.5 0.5
Claire 0.9 0.1

Cycling Fishing Running Tango Salsa Waltz

• Binary latent Alice 1 1 1

features… Bob 1 1
Claire 1 1

15
Latent Variable Models
As Matrix Factorization

16
Social network models
as matrix factorization

17
 Learning outcomes
By the end of the lesson, you should be able to:

• Model social networks by applying the principles

of latent variable modeling you have already
learned in this course:
– Generative probabilistic models with latent variables
– Latent clusters, mixed membership,
latent linear models, GLMs
– Bayesian inference

18
19
20
21
 Stochastic blockmodels
(Nowicki and Snijders, 2001)
• Probabilistic models which

– Try to find latent groups of nodes

– Latent variables zi are cluster assignments

– Connections between nodes depend on cluster

assignments

22
University email network:
Note the block structure

23
 Stochastic equivalence
(a.k.a. structural equivalence)

• Members of a group have similar relationship patterns

24
 Stochastic equivalence
vs community detection
Original graph Blockmodel

Stochastically equivalent, but

are not densely connected

Figure due to Goldenberg et al. (2009) - Survey of Statistical Network Models, Foundations and Trends 25
 Stochastic blockmodel
latent representation
Alice Bob

Claire

UCSD UCI UCLA

Alice 1

Bob 1

Claire 1
26
Reordering the matrix to show the
inferred block structure

27
Kemp, Charles, et al. "Learning systems of concepts with an infinite relational model." AAAI. Vol. 3. 2006.
 Model structure
Interaction matrix W

Latent blocks Z (probability of an

edge from block k to
block l)

28
Kemp, Charles, et al. "Learning systems of concepts with an infinite relational model." AAAI. Vol. 3. 2006.
Stochastic blockmodel
generative process

29
 The Infinite Relational Model (IRM)
Kemp et al., (2006)
• Multiple relations modeled simultaneously, based on
the same latent clusters
• Multiple types of entities (modes)
• Learn number of blocks/clusters automatically
Social relations
(likes, admires,
respects, hates,…) Demographics

People

Questions on a
People personality test 30
 The Infinite Relational Model (IRM)
Kemp et al., (2006)
• Each entity assigned to a cluster
• Each relation is conditionally independent given Z
• Parameter matrix (or tensor) W(r) for each relation
• W(r) has as dimensions = arity of r
Social relations
(likes, admires,
respects, hates,…) Demographics

People

Questions on a
People personality test 31
 32
Kemp, Charles, et al. "Learning systems of concepts with an infinite relational model." AAAI. Vol. 3. 2006.
Mixed membership stochastic blockmodels
(Airoldi et al., 2008)

Alice Bob

Claire

Running Dancing Fishing

Alice 0.4 0.4 0.2

Bob 0.5 0.5

Claire 0.1 0.9

33
Mixed membership stochastic blockmodels
(Airoldi et al., 2008)

34
 Application of MMSB to
Sampson’s Monastery

Original network Summary Denoising (use z’s)

(use π‘s)
Airoldi, E. M., Blei, D. M., Fienberg, S. E., & Xing, E. P. (2009). Mixed membership stochastic blockmodels. 35
In Advances in Neural Information Processing Systems (pp. 33-40).
Latent Feature Relational Model
Miller, Griffiths, Jordan (2009)

Alice Bob

Cycling Tango
Fishing Salsa
Running
Claire

Waltz
Running
Latent Feature Relational Model
Miller, Griffiths, Jordan (2009)

Alice Bob

Cycling Tango
Fishing Salsa
Running
Claire

Waltz
Running
Latent Feature Relational Model
Miller, Griffiths, Jordan (2009)

Alice Bob

Cycling Tango
Fishing Salsa
Running
Claire

Waltz
Running
 Latent Feature Relational Model
Miller, Griffiths, Jordan (2009)

Alice Bob

Cycling Tango
Fishing Salsa
Running
Claire

Waltz
Running

Mixed membership implies a kind of “conservation of (probability) mass” constraint:

If you like cycling more, you must like running less, to sum to one
 Latent Feature Relational Model
Miller, Griffiths, Jordan (2009)

Alice Bob

Cycling Tango
Fishing Salsa
Running
Claire

Waltz
Running

Cycling Fishing Running Tango Salsa Waltz

Alice
Z= Bob
Claire
 Latent Feature Relational Model
Miller, Griffiths, Jordan (2009)
• Likelihood model:

“If I have feature k, and you have feature l, add Wkl to

the log-odds of the probability we interact”

• Can include terms from p2 model for network

density, covariates, popularity,…
41
Dynamic Relational Infinite Feature Model
(DRIFT) (Foulds et al., 2011)
• Models networks as they over time, by way of
changing latent features

Alice Bob

Cycling Tango
Fishing Salsa
Running
Claire

Waltz
Running

42
Dynamic Relational Infinite Feature Model
(DRIFT) (Foulds et al., 2011)
• Models networks as they over time, by way of
changing latent features

Alice Bob

Cycling Tango
Fishing Salsa
Running
Claire

Waltz
Running

43
Dynamic Relational Infinite Feature Model
(DRIFT) (Foulds et al., 2011)
• Models networks as they over time, by way of
changing latent features

Alice Bob

Cycling Tango
Fishing Salsa
Running
Claire

Waltz
Running

44
Dynamic Relational Infinite Feature Model
(DRIFT) (Foulds et al., 2011)
• Models networks as they over time, by way of
changing latent features

Alice Bob

Cycling Tango
Fishing Salsa
Running Fishing
Claire

Waltz
Running

45
Dynamic Relational Infinite Feature Model
(DRIFT) (Foulds et al., 2011)
• Models networks as they over time, by way of
changing latent features

Alice Bob

Cycling Tango
Fishing Salsa
Running Fishing
Claire

Waltz
Running

46
 Dynamic Relational Infinite Feature Model
(DRIFT) (Foulds et al., 2011)

• HMM dynamics for each actor/feature (factorial HMM)

• Inference via MCMC: forward/backward
47
 Homophily:
“Birds of a feather flock together”

48
 Latent space model
(Hoff et al., 2002)
• Embed nodes in a continuous
latent space

• Probability of a link depends on

distance in the latent space

• Metropolis-Hastings inference. Initialize at MLE

49
Application to Sampson’s monastery

MLE solution MCMC posterior samples

P. D. Hoff, A. E. Raftery, and M. S. Handcock, “Latent space approaches to social network analysis,” 50
Journal of the American Statistical Association, vol. 97, no. 460, pp. 1090–1098, 2002.
51
52
 Think-pair-share:
International relations
• You have data on the interactions between pairs of
countries, consisting of the number of cooperative
and conflictual interactions from each country a to
each country b in the last 12 months.

• Design a latent variable model to visualize

interpretable patterns in this dataset and predict
missing edge-counts.

53