PROGRAMMING

in the
LARGE
!
B
"
z
@markhibberd
!
B
"
z
SYSTEM
FORMULATION
How do we arrive at
Systems?
Modernization
Old New
Productionization
Prototype Production
Consolidation
Bigger
Small
Small
Modularization
Big
Smaller
Smaller
Greenfield
Legacy
Incremental
Base More More
THESE ARE NOT
SYSTEMS
Any organization that designs a
system will produce a design whose
structure is a copy of the organization's
communication structure
Conways Law
The rules and boundaries
of our systems should
stem from architecture
not organisation
How the system interacts
What is the systems purpose
Macro Architecture
Domain Architecture
How the system is built
Micro Architecture
Systems as the unit of work
Not unit of work as a system
!
B
"
z
SYSTEMS
of
SYSTEMS
The thing about real systems is
AUTONOMY
CROSS LANGUAGE CODE SEARCH
AS AN EXAMPLE
Code search the JVM wrong way
Pick Framework
Scala + Play
Pick Language + Framework
Code search the JVM wrong way
UI API
FILESYSTEM
Code search the JVM wrong way
Scala + Play
UI API
DB INDEXER
Code search the JVM wrong way
Scala + Play
SYSTEMS, NOT SUB-SYSTEMS
Code search as Systems
INDEXING SEARCH
Code search as Systems
D
o
m
a
i
n
A
r
c
h
i
t
e
c
t
u
r
e
INDEXING SEARCH
Code search as Systems
INDEXING SEARCH
CODE
CTAGS
CTAGS
application/html
application/search.v1+json
Code search as Systems
INDEXING SEARCH
CODE
CTAGS
CTAGS
application/html
application/search.v1+json
M
a
c
r
o
A
r
c
h
i
t
e
c
t
u
r
e
STANDARD FORMATS HELP
GIVE US AUTONOMY
Code search as Systems
INDEXING SEARCH
Scala
Embedded Server
OS for logging
Bourne Shell
Exuberant CTAGS
Deploy as git hook
OS for logging
OS for services
Code search as Systems
INDEXING SEARCH
M
i
c
r
o
A
r
c
h
i
t
e
c
t
u
r
e
Scala
Embedded Server
OS for services
OS for logging
Bourne Shell
Exuberant CTAGS
Deploy as git hook
OS for logging
Code search as Systems
INDEXING SEARCH
M
i
c
r
o
A
r
c
h
i
t
e
c
t
u
r
e
Scala
Embedded Server
OS for services
OS for logging
Bourne Shell
Exuberant CTAGS
Deploy as git hook
OS for logging
MORE AUTONOMY
RULES
ARCHITECTURE IS ABOUT
NOT BOXES
Signs of failure
Systems are not autonomous
Systems share a domain model
Systems need to be evolved in step
Systems need to be built by single team
Systems share a data base
Systems as the unit of work
If you cant add a person to
your project then you are
failing worse than you think
again
An Aside
!
B
"
z
EVOLUTION
Thinking ahead is not about avoiding change
Thinking ahead is about letting us change at
different rates for different problems
Indexing V2
INDEXING SEARCH
Scala
Embedded Server
OS for logging
Haskell
Static Executable
OS for services
OS for logging
OS for services
INDEXING SEARCH
Scala
Embedded Server
OS for logging
Haskell
Static Executable
OS for services
OS for logging
OS for services
M
i
c
r
o
A
r
c
h
i
t
e
c
t
u
r
e
Indexing V2
now vs later
priorities shift over time,
can't let early traction
sacrifice long term speed
An Aside
INDEXING SEARCH
Scala
Embedded Server
OS for logging
Haskell
Static Executable
OS for services
OS for logging
OS for services
Indexing V2
DIFFERENT SYSTEMS SAME RULES
DIFFERENT SYSTEMS DIFFERENT RULES
An Aside
M
a
c
r
o
A
r
c
h
i
t
e
c
t
u
r
e
Indexing V3
INDEXING SEARCH
CODE
CTAGS
CTAGS
application/html
application/search.v1+json
Super-Zippy Format
M
a
c
r
o
A
r
c
h
i
t
e
c
t
u
r
e
Search V2
INDEXING SEARCH
CODE
CTAGS
Super-Zippy Format
application/html
application/search.v1+json
Super-Zippy Format
Notice how easy it is to think of
evolution in the large
Rates of change
domain
macro
micro
Rates of change
domain
macro
micro
Freedom to choose tools and technology by succeeding at domain and protocols
HOW DO SYSTEMS THINK
HOW DO SYSTEMS COMMUNICATE
HOW DO SYSTEMS CHANGE
Implications at
Scale
The concept of micro-services are great
An Aside
The concept of small libraries are great
But they are not enough without
thinking in systems
Larger Systems of Systems
How long does it take to get 1 line
of code to production
Larger Systems of Systems
Mistaking Modules For Systems
Mistaking Modules For Systems
Mistaking Modules For Systems
Implication of Autonomy
USERS
Implication of (lack of ) Autonomy
USERS
!
B
"
z
SYSTEMS
in
ACTION
CMSs hold customer data hostage
CONTENT ANALYSIS & DATA MINING PROJECT
Free data
Analyse data
Derive data
CMSs tied to horrible platforms
Goals
Constraints
CONTENT ANALYSIS & DATA MINING PROJECT
Prototype Production
1
CONTENT ANALYSIS & DATA MINING PROJECT
Prototype Production
1
JSP Java
CONTENT ANALYSIS & DATA MINING PROJECT
Prototype Production
1
1 JSP Java
CONTENT ANALYSIS & DATA MINING PROJECT
Prototype Production
Smaller Smaller
1
2
Java 1 JSP
CONTENT ANALYSIS & DATA MINING PROJECT
Prototype Production
Smaller Smaller
1
2
1 JSP Java
Java Scala
WHAT I IMAGINED
DATA
WEB
CORE LEECH PORTAL
CONTENT ANALYSIS & DATA MINING PROJECT
WEB / CORE / DATA / LEECH
WEBSHERE
REALITY SETTING IN
CAUGHT THINKING IN THE
SMALL
MODULES - LIBRARIES - TECH
We should leverage architecture changes
rather than force new technology
CONTENT
ANALYSIS
IDENTITY
CONTENT
STORE
QUERY
ENGINE
A Better Approach
Play 1.1 App
Our Starting Point
Play 1.1 App
Enabling Moves
IDENTITY
Play 1.1 App
Protobufs
Thrift
HTTP
Establishing Rules & Autonomy
sbt-assembly
OS package
Containers
IDENTITY
Protocols
Deployment
Play 1.1 App
Identifying Demand
CONTENT
STORE
IDENTITY
Play 1.1 App
Validation
CONTENT
STORE
IDENTITY
Play 1.1 App
Repeat
CONTENT
STORE
IDENTITY
CONTENT
ANALYSIS
Repeat
CONTENT
STORE
IDENTITY
CONTENT
ANALYSIS
QUERY
ENGINE
architecture is an everyday task
good architecture means never
having to rewrite
All editors are bad, really bad
ONLINE CONTENT EDITING
Something aaS
APIs
Ready to change
Small team, tight deadline
Goals
Constraints
Lots of ideas, not sure what would work
Spelling Identity Templates
Editor
Customer
UI
D
o
m
a
i
n
A
r
c
h
i
t
e
c
t
u
r
e
Spelling Identity Templates
Editor
Customer
UI
M
a
c
r
o
A
r
c
h
i
t
e
c
t
u
r
e
HTTP / JSON
C Scala Scala
Spelling Identity Templates
Editor
Customer
UI
M
i
c
r
o
A
r
c
h
i
t
e
c
t
u
r
e
Scala/JS Scala
Postgres
Postgres Postgres
syslog
jail per service
rc.d
1 executable
Minimal stack
Editor was working well
ONLINE CONTENT EDITING
Incremental
Multiple Versions
Add not modify
Needed to add more services
Goals
Constraints
System need to keep running
Spelling Identity Templates Content
Editor Analysis
Customer
UI
Spelling Identity Templates Content
Editor Analysis
Customer
UI
HTTP / JSON
Spelling Identity Templates Content
Editor Analysis
Customer
UI
Scala/JS Scala JS
C Scala Scala Scala
multilog
jail per service
daemon tools
1 executable
Minimal stack
Demand for standalone features
ONLINE CONTENT EDITING
Quick
Minimal Overhead
Maximum Reuse
Not tied to our editor
Goals
Constraints
Run in parallel to current system
Spelling Identity Templates Content
Editor Analysis
Customer
UI
UGLY BITS
Spelling Identity Templates Content
Editor Analysis
Customer
UI
Glue
Glue Glue Glue
Glue Glue
Glue
Spelling Identity Templates Content
Editor Analysis
Customer
UI
Config
ARCHITECTURE SHOULD HAVE
RULES FOR HOW WE SHARE CODE
!
B
"
z
RECAP
RULES
ARCHITECTURE IS ABOUT
NOT BOXES
good architecture means never
having to rewrite
Thinking ahead is not about avoiding change
Thinking ahead is about letting us change at
different rates for different problems
Systems as the unit of work
Not unit of work as a system
The thing about real systems is
AUTONOMY
!
B
"
z
FIN
@markhibberd
References + More Info
Stefan Tilkov's Breaking the Monolith slides video
Coda Hale and Ryan Kennedy on "Streamie" notification service at yammer
Slide Deck
v2 v1