High performance computing for non-programmers

Glib Ivashkevych
junior researcher, NSC KIPT

Lecture #12: Data structures

Data structs & algorithms rationale

Good algorithm can improve runtime performance by orders of magnitude Algorithms (and programs in general) operate on data structures Data structures + Algorithms = Programs

Algs: spoiler O-notation

How to measure and describe algorithm performance? We can: measure or analyze Asymptotic running time matters T(N) ~ O(N2), say O(N) is better, than O(N2) O(NlogN) is better, than O(N2)

Data structures compound d.s.

Array
contiguous space in memory holding elements of the same size

Linked list
collection of linked elements of, prob, different size Array = random access, costly to reshape

Linked list = sequential access, easily reshaped

Data structures arrays

item0

1
//int arr[num] = {0};

int arr[num]; int *arr;

arr = (int*) malloc(num); free(arr);

//somewhere

//calloc(num, sizeof(int))

Data structures arrays

Pros: O(1) access time compactness (no extra data stored) cache friendly Cons: reshape operations are costly (reallocation needed)

Data structures linked lists

data data data data

first

last

To implement linked list one needs structs (or classes in C++).

Data structures linked lists

Pros: easily to reshape (no cost) and insert elements Cons: traversal is costly (O(N)) not cache friendly memory overhead

Data structures abstract d.s.

Stack Last-In-First-Out collection Queue First-In-First-Out collection Tree hierarchical collection

Data structures stack

Stack operations: push item to stack
in last out

item is added on top of the stack pop item from stack item is removed

first

from top of the stack

Data structures stack

Applications: expression evaluation
4+8*2 +4 (* 8 2) (stacked from R to L) or 4+8*2 4 (8 2 *) + 4 8 2 * + (stacked from L to R)

memory management + virtual machines

(both Python VM and Java VM use stacks, although in different ways)

Data structures stack

Implementations: Python: list C++: std::stack, boost::lockfree::stack Java: java.util.Stack

Data structures queue

Queue operations: enqueue item
first out

item is added at the end of the queue dequeue item item is removed

in

last

from the beginning of the queue

Data structures queue

Applications: many:) processes queue in OS message queues In general: serve multiple requests to single shared resource

Data structures queue

Implementations: Python: list:) C++: std::queue, boost::lockfree::queue Java: java.util.Queue interface
(with many implementing classes)

Data structures priority queue

Max element is removed, instead of oldest
PQueue operations: enqueue item
first max out

dequeue max item with largest key is removed Many approaches: (un-)ordered seq-s, heaps Best you can get: logN

in

last

Data structures dequeue

Queue operations: enqueue front
in first out

enqueue end dequeue front dequeue end Applications:

in

last

out

Some scheduling algorithms Undo-redo, history

Questions?