The Graphics Pipeline and OpenGL

Prof.Vladlen Koltun Computer Science Department Stanford University

The synthetic camera model

Two components of viewing - Set of geometric objects that form the content of the scene - Viewer through which the scene is imaged

The graphics pipeline

Primitives Geometry processor Rasterizer Fragment processor Frame buffer

Primitives Geometry processor Rasterizer Fragment processor Frame buffer

The geometry processor

Transforms primitives to the cameras coordinate system, prepares them for rasterization Culls primitives facing away from the camera or lying outside the view frustum

Primitives Geometry processor Rasterizer Fragment processor Frame buffer

The rasterizer

Generates fragments (proto-pixels)

Primitives Geometry processor Rasterizer Fragment processor Frame buffer

Fragment processor

Checks if fragments are visible Determines color All fragments treated identically, irrespective of the original primitive

Primitives Geometry processor Rasterizer Fragment processor Frame buffer

Frame buffer

Memory buffer used for the construction of the image. Not all data that passes through the frame buffer is displayed. It is like a sandbox in which the image is constructed. Used by the window system for display.

Primitives Geometry processor Rasterizer Fragment processor Frame buffer

Double buffering
Rendering Window system

Back buffer

Front buffer

Render into the back buffer while the window system points to the front buffer. When the next frame is assembled, swap. Avoids terrible visual artifacts

Primitives Geometry processor Rasterizer Fragment processor Frame buffer

Double buffering
Rendering Window system

Front buffer

Back buffer

Render into the back buffer while the window system points to the front buffer. When the next frame is assembled, swap. Avoids terrible visual artifacts

Primitives Geometry processor Rasterizer Fragment processor Frame buffer

Advantages and disadvantages of pipeline model

Great for parallel processing Primitives processed independently Fragments processed independently Does not support interactions between multiple objects in the scene Global illumination, shadows, reection, refraction

The pipeline is evolving

Blythe, The Direct3D 10 System, SIGGRAPH 2006

Alternatives to the pipeline?

Global illumination

Consider indirect illumination that is transmitted by means of other objects Primitives are no longer independent

Ray tracing

Rays are cast from the viewpoint and followed recursively through the scene. Whitted ray tracing: Compute direct illumination from light sources at every point hit by traced rays
CC image from Wikipedia (author: Mimigu)

Radiosity

Discretize scene into patches. Compute strength of interaction between patches. Shoot light from source patches, deposit in other patches. Iterate until light is absorbed.
Cornell University Program of Computer Graphics

Photon mapping

Stage 1: Trace photons from light sources and deposit onto photon map when photons interact with diffuse surfaces Stage 2: Cast rays from viewpoint and estimate radiance

OpenGL

Why a graphics API?

Why a graphics API?

Save developers from having to implement standard functionality Facilitate portability through abstraction

#include <GL/glut.h> void display() { glClear(GL_COLOR_BUFFER_BIT); glBegin(GL_POLYGON); glVertex2f(-0.5, -0.5); glVertex2f(-0.5, 0.5); glVertex2f(0.5, 0.5); glVertex2f(0.5, -0.5); glEnd(); glFlush(); } void init() { glClearColor (0.0, 0.0, 0.0, 0.0); glColor3f(1.0, 1.0, 1.0); glMatrixMode (GL_PROJECTION); glLoadIdentity (); glOrtho (-1.0, 1.0, -1.0, 1.0, -1.0, 1.0); }

Code sample

int main(int argc, char** argv) { glutInit(&argc,argv); glutCreateWindow("simple"); glutDisplayFunc(display); init(); glutMainLoop(); }

Design considerations for OpenGL

Enable applications to run on a variety of platforms Describe rendering operations succinctly Accommodate extensions as graphics hardware evolves Maximize performance by closely modeling the capabilities and characteristics of graphics hardware

Agnostic to the window system

OpenGL was designed to work with a variety of window systems - Interaction with the window system is handled by GLUT No facilities for obtaining user input Does not handle the actual display, merely assembles an image

A rendering API

The OpenGL API supports the display of geometric primitives - Takes a specication of geometric objects - Computes illumination - Images the scene Modeling and animation largely left to the application

Separation of content and viewer

Separates object description from viewer specication Two types of functions - Describe objects in the world (the input) - Specify how the objects should be processed for constructing an image (the state)

OpenGL is a state machine

State machine with inputs and outputs - Input is geometric objects, output is a set of pixels - State machine converts a collection of geometric objects in three dimensions to an image. This process is controlled by the state. - State species how objects are projected onto the image plane, how the are colored, etc.

Segal and Akeley, The Design of the OpenGL Graphics Interface, 1994

EnableClientState DisableClientState

EdgeFlagPointer TexCoordPointer ColorPointer IndexPointer NormalPointer VertexPointer InterLeavedArrays Evaluator EvalMesh ArrayElement DrawElements DrawArrays Map Evaluation EvalCoord EvalPoint Grid Application Control Vertex Array Control MapGrid

The OpenGL Machine

R

The OpenGL! graphics system diagram, Version 1.1. Copyright " 1996 Silicon Graphics, Inc. All rights reserved.
Map Enable/Disable

EdgeFlag

Current Edge Flag Enable/Disable

TexCoord1

TexGen OBJECT_LINEAR TexGen EYE_LINEAR TexGen SPHERE_MAP b A A*b

TexCoord2

TexCoord3

1 Current TexGen

Texture Matrix Stack

TexCoord4

Texture Coordinates

Vertices
Evaluators & Vertex Arrays
Material Parameters Control

Color3

1 Enable/Disable Current RGBA Color ColorMaterial Material

Color4

Convert RGBA to float

LightModel Begin/End

Index

Convert index to float

Current Color Index

Light Enable/Disable

Light Parameters

Material Parameters

Light Model Parameters

Enable/Disable Clamp to [0,1] Mask to n1 [0,2 ] Primitive Assembly

Input Conversion & Current Values

Texture Coordinate Generation

Lighting

RGBA Lighting Equation Normal3 Convert normal coords to float Current Normal b M Color Index Lighting Equation M*b Normalize

Vertex2 RasterPos2

MT M FrontFace

Enable/Disable (Lighting)

Matrix Control

Vertex3 RasterPos3

Clipping, Perspective, and Viewport Application Feedback & Selection

Rasteriz ation

Vertex4 RasterPos4

OBJECT COORDINATES

M M

M*b

EYE COORDINATES

Texturing, Fog, and Antialiasing

PerFragment Operations Frame Buffer & Frame Buffer Control

Pixels

Rect

Rectangle Generation

Primitives
Model View Matrix Stack

Fragments

Key to OpenGL Operations

Enable/Disable (Antialiasing/Stipple) ClipPlane FrontFace CullFace M M b T b Polygon Culling Polygon Polygon Mode Enable/Disable (Antialiasing) LineStipple LineWidth Divide Vertex Coordinates by w Line Segment Rasterization TexParameter Enable/Disable (Antialiasing) PointSize Enable/Disable TexEnv Enable/Disable Fog Enable/Disable Enable/Disable Enable/Disable Scissor Coverage (antialiasing) Application Enable/Disable AlphaFunc StencilOp StencilFunc Enable/Disable DepthFunc Enable/Disable BlendFunc Enable/Disable Enable/Disable LogicOp Rasterization PolygonMode PolygonOffset

MatrixMode PushMatrix PopMatrix

Matrix Control

LoadIdentity LoadMatrix M*N N M Clip Planes Projection Matrix Stack Viewport DepthRange

ShadeModel

MultMatrix Translate Scale Rotate Frustum Ortho Matrix Generators

POLYGONS
Flatshading

Polygon Clipping

M M*b

Polygon View Volume Clipping Line View Volume Clipping Point View Volume Culling

LINE SEGMENTS POINTS RASTER POS.

Line Clipping Point Culling

M*b (Vertex Only)

Apply Viewport

M*b

Point Rasterization Current Raster Position RenderMode

Texel Generation

Texture Application

Fog

Pixel Ownership Test

Scissor Test

Alpha Test (RGBA only)

Stencil Test

Depth Buffer Test

Blending (RGBA only)

Dithering

Logic Op

Notes: 1. Commands (and constants) are shown without the gl (or GL_) prefix. 2. The following commands do not appear in this diagram: glAccum, glClearAccum, glHint, display list commands, texture object commands, commands for obtaining OpenGL state (glGet commands and glIsEnabled), and glPushAttrib and glPopAttrib. Utility library routines are not shown. 3. After their exectution, glDrawArrays and glDrawElements leave affected current values indeterminate. 4. This diagram is schematic; it may not directly correspond to any actual OpenGL implementation.

Clear Selection Encoding Selection Control Feedback Encoding Bitmap Rasterization

Clear Control Masking

PassThrough SelectBuffer Selection Name Stack FeedbackBuffer

PolygonStipple Bitmap DrawPixels TexImage Unpack Pixels Pixel Transfer

PixelZoom

Clear Values

DepthMask StencilMask

Pixel Rasterization Texture Memory

ClearStencil ClearDepth ClearIndex ClearColor Masking Frame Buffer Control

InitNames LoadName PopName PushName

TexSubImage PixelStore PixelTransfer PixelMap

Frame Buffer

ColorMask IndexMask

DrawBuffer Readback Control

ReadPixels

Pack Pixels

CopyPixels CopyTexImage CopyTexSubImage

ReadBuffer

Segal and Akeley, The OpenGL Graphics System: A Specication (Version 1.1)

Declarative vs. Imperative programming

OpenGL is imperative - Operations specify how rendering should be conducted (by changing the state) - Operations do not execute individual rendering operations In declarative APIs (cf. RenderMan), the application commands the graphics system what to do, not just how to do what it does The OpenGL abstraction controls what appears to happen, not what actually happens. Leaves signicant freedom for optimization by OpenGL implementations.

Client vs. Server

Application
Rendering client Narrow pipe

Graphics hardware
Rendering server

Central assumptions: The client is computationally weak, the server is computationally powerful Bandwidth is low

Client vs. Server

Application
Rendering client Narrow pipe

Graphics hardware
Rendering server

Examples Color is part of the state, not associated with the primitive. When a new transformation matrix is specied, the default behavior is to multiply, rather than replace

Persisting primitives

Display lists Vertex buffer objects