Professional Documents
Culture Documents
(Greek letters are written out so this can be read as plain text.)
Contents
Machine Finite Automata, FA, DFA, FSM Nondeterministic Finite Automata, NFA Pushdown Automata, PDA Turing Machine Universal Turing Machine Church Turing Hypothesis Languages accepted by polynomial time Turing Machines, P Nondeterministic Polynomial time Turing Machines, NP Moore Machine Mealy Machine Machine Equivalences Machine to Language Equivalences Equivalent Computational Power Other Links
Machine
A general term for an automata. A machine could be a Turing Machine, a pushdown automata, a finite state machine or any other restricted version of a Turing machine.
Finite Automata also called a Finite State Machine, FA, DFA or FSM
M = (Q, Sigma, delta, q0, F) is a definition of a Finite Automata. To be fully defined the following must be provided: Q a finite set of states often called q0, q1, ... , qn or s0, s1, ... , sn. There is no requirement, in general, that every state be reachable. sigma a finite input alphabet
delta a transition function mapping Q cross sigma to Q. Alpha is typically given as a table with all states listed on the right, input symbols listed across the top and next state in the table: input delta, the transition table | 0 | 1 | ----+-----+-----+ s0 | s1 | s2 | ----+-----+-----+ s1 | s2 | s0 | ----+-----+-----+ s2 | s2 | s1 | ----+-----+-----+
state
When the transition table, delta, has all single entries, the machine
may be refereed to as a Deterministic Finite Automata, DFA. There is no requirement, in general, that every entry in the table must contain a state. If a machine tries to go to an empty table entry it "crashes". The remainder of the input, if any, is unread. The machine does not accept and can not possibly be in a final state. When every state/input pair has at most one target state, the automata is called deterministic. (See below for definition of nondeterministic.) q0 the initial state from the set Q. By definition this is the state the automata is in when it starts. The automata gets the first symbol from the input, then goes from the starting state to the state designated by the transition function. F a set of final states from the set Q. Final states are also known as accepting states. The machine stops after the last input symbols is read and the corresponding state transition occurs. If the machines state when stopped is in F then the machine is said to accept the input string. F can be a null set in which case only the empty language is accepted. There is no requirement, in general, that any final state be reachable. A machine defines a language, the set of all strings accepted by the machine. This language is usually denoted L(M). The machine that accepts a language L is usually denoted M(L). There is a Finite Automata, as defined here, for every Regular Language and a Regular Language for every Finite Automata. Another common way to define a Finite Automata is via a diagram. The states are shown as circles, often unlabeled, the initial state has an arrow pointing to it, the final states have a double circle, the transition function is shown as directed arcs with the input symbol(s) on the arc.
state
A string is accepted if any sequence of transitions ends in a Final state. There could be more than one sequence of transitions that end in a Final state. Think of each transition that has more than one state as causing a tree to branch. All branches are in some state and all branches transition on every input. Any branch that reaches phi, the null or nonexistent state, terminates. Any NFA can be converted to a DFA but the DFA may require exponentially more states than the NFA.
The ordered pairs (q sub i, gamma sub i) are respectively the next state and the string of symbols to be written onto the stack. The machine is nondeterministic, meaning that all the pairs are executed causing a branching tree of PDA configurations. Just like the branching tree for nondeterministic finite automata except additional copies of the pushdown stack are also created at each branch. The operation of the PDA is to begin in state q0, read the symbol on the input tape or read epsilon. If a symbol is read, the read head moves to the right and can never reverse to read that symbol again. The top of the stack is read by popping off the symbol. Now, having a state, an input symbol and a stack symbol a delta transition is performed. If there is no delta transition defined with these three values the machine halts. If there is a delta transition with the (q,a,A) then all pairs of (state,gamma) are performed. The gamma represents a sequence of push down stack symbols and are pushed right to left onto the stack. If gamma is epsilon, no symbols are pushed onto the stack. Then the machine goes to the next state, q. When the machine halts a decision is made to accept or reject the input. If the last, rightmost, input symbol has not been read then reject. If the machine is in a final state accept. If the set of final states is empty, Phi, and the only symbol on the stack is Z0, then accept. (This is the "accept on empty stack" case) Now, using pictures we show the machines for DFA, PDA and TM +-------------------------+----------------- DFA, NFA, NFA epsilon | input string | accepts Regular Languages +-------------------------+----------------^ read, move right | | +-----+ | | |--> accept +--+ FSM | M = ( Q, Sigma, delta, q0, F) | |--> reject +-----+ +-------------------------+----------------- Push Down Automata | input string |Z0 stack accepts Context Free Languages +-------------------------+----------------^ read, move right ^ read and write (push and pop) | |
+-----------------------+ | +-----+ | | |--> accept +--+ FSM | Z0, F) | |--> reject +-----+
+-------------------------+----------------- Turing Machine | input string |BBBBBBBB ... accepts Recursively Enumerable +-------------------------+----------------- Languages ^ read and write, move left and right | | +-----+ | | |--> accept +--+ FSM | M = ( Q, Sigma, Gamma, delta, q0, B, F) | |--> reject +-----+ An example of a language that requires the PDA to be a NPDA, Nondeterministic Push Down Automata, is L = { w wr | w in Sigma and wr is w written backwards }
Turing Machine, TM
A Turing machine, named after A. Turing, is defined by M = (Q, Sigma, Gamma, delta, q0, B, F) where is a finite set of states (same as DFA) is a finite set of tape symbols is a symbol in Gamma is the blank, or space symbol is a subset of Gamma not including B, the input symbols is the transition function, next move table, that maps Q cross Gamma to Q cross Gamma cross {Left, Right} q0 is in Q and is the initial or starting state F is a subset of Q, the final states. After the last input symbol has been read and the tape is in the leftmost position, a string is accepted if M is in a final state. In this particular model, the tape is only unbounded on the right, filled with B's and fixed on the left at the first input symbol. Other models include a tape also unbounded on the left , symbols on the tape for "left-end" and "right-end", ability to write no symbol, ability to move {Left, Right or None}. These variations and many more are provably equivalent. A Turing Machine that computes a value rather than recognizing a language, leave the computed value on the tape and halts. It is possible for a TM to never halt on some input strings that are not in the language it accepts or on input values for which it is not a complete function. Q Gamma B Sigma delta
The set of all Turing machines is countable. Thus there exists real numbers that can not be accepted by any Turing machine. delta transition table might look like
input delta, the transition table | 0 | 1 | b | ----+----------+----------+----------+ s0 | (s1,1,R) | (s2,0,L) | Phi | ----+----------+----------+----------+ s1 | (s0,0,L) | Phi | (s0,1,R) | ----+----------+----------+----------+ s2 | Phi | (s1,1,R) | (s2,b,L) | ----+----------+----------+----------+
state
A Turing machine may also have a nondeterministic transition table that has sets of entries. Every member of the set of a transition then continues operation in parallel, each with its own state and tape. Phi is it is "hung" no accept occurs the null or nonexistent state. If a TM enters the phi state and there is no indication to the outside world. There is
and no reject output signal. No further tape head movement and no further symbols are read from the tape.
Each "step" executed by a Turing machine does the following: The machine is in some state, q0 initially. The read/write head is positioned on some tape cell, initially leftmost. The pair (state q, input symbol a) select one cell in the delta transition table. e.g. delta(q,a) = (qi, b , R) The machine enters state qi, writes b onto the tape, moves right. The running time of a Turing machine is measured in number of steps and is usually related to the length of the input, the number of symbols on the initial tape.
One UTM can simulate another UTM that is simulating a TM with an input string. Etc. Etc. Etc.
Moore Machine
A Moore machine is defined as M = (Q, sigma, Delta, delta, gamma, q0) Where Q, Sigma, delta and q0 are as defined for finite automata. Delta is the output alphabet. gamma is a mapping from Q to Delta. +-------------------------+ | input string | Moore machine, no more powerful than DFA +-------------------------+ ^ read, move right | | +-----+ | | | +--+ FSM | M = ( Q, Sigma, Delta, delta, gamma, q0) | | +-----+ | v +----------------------------| output tape alphabet Delta (May be same or vary from Sigma) +----------------------------Example: Q= {q0, q1} q1->B Sigma= {a, b} Delta= {A, B} gamma= q0->A,
delta | a | b | gamma ------+----+----+-----q0 | q1 | q0 | A q1 | q0 | q1 | B A B <--shows symbol written to output when entering state _____ a _____ --> / q0 \----->/ q1 \ and other connecting transitions like a DFA
\_____/<---- \_____/ b This is a finite state machine that outputs a symbol on every state transition starting with gamma(q0) which is always output. Note there is no set F of final states. A Moore machine is equivalent to a DFA when Delta = {0, 1} and a zero output is interpreted as reject, while a 1 output is interpreted as accept and gamma maps final states to 1 and all other states to 0. A finite automata can only accept (last input symbol put machine in a final state) or reject an input string. A Moore machine is like a simple function where an input is processed and a result is produced. Moore machines are usually not based on NFA's because this would be like a multi valued function. Some input could produce more than one output.
Mealy Machine
A Mealy machine is defined as M = (Q, sigma, Delta, delta, gamma, q0) using the definitions from the Moore machine with the exception that gamma maps Q cross Sigma to Delta. Delta is the output alphabet. Thus the output is a function of the state and the input symbol, written during the state transition. The difference from a Moore machine is the output being a function of the transition rather than just a function of the sequence of states. The empty string as input produces an empty output. +-------------------------+ | input string | Mealy machine, no more powerful than DFA +-------------------------+ ^ read, move right | | +-----+ | | | +--+ FSM | M = ( Q, Sigma, Delta, delta, gamma, q0) | | +-----+ | v +----------------------------| output tape alphabet Delta (May be same or vary from Sigma) +----------------------------Example: Q= {q0, q1} in delta delta | a | b Sigma= {a, b} Delta= {A, B, C, D} gamma is
------+------+-----q0 | q1A | q0C q1 | q0D | q1B /--shows symbol written to output upon transition _____ a/A _____ --> / q0 \------->/ q1 \ and other connecting transitions state/output \_____/<------ \_____/ a/D Mealy machines are usually not based on NFA's because this would be like a multi valued function. Some input could produce more than one output. The proof that a Mealy machine and a DFA are equivalent, using gamma being a one for accept on transition to a final state, and gamma being zero otherwise, also needs a little effort to have a Mealy machine accept the null string, epsilon.
Machine Equivalences
The following machines are provably equivalent FA = DFA = NFA = NFA epsilon FA = two way FA (tape can move left, right, or not move) FA = Moore machine with output alphabet {0, 1} and state q accepting if and only if gamma(q)=1 FA = Mealy machine with some technicalities TM TM TM TM TM = = = = = unrestricted grammar lambda calculus Post formal systems NTM TM with more tapes, write or not write, move or not move, etc.
Various Equivalences
The following group can have equivalent computational power partial recursive functions r.e. languages = recursively enumerable languages Turing machines that may not halt when input is not accepted unrestricted grammars lambda calculus Post Formal Systems computer programs in a reasonable language that may not halt on some input Chomsky type 0 languages
The following group can have equivalent computational power total recursive functions recursive languages = rec Turing machines that halt on all inputs grammars that stop on all inputs lambda calculus with finite recursion computer programs that halt on all inputs The following group can have equivalent computational power context sensitive languages, CSL's context sensitive grammars, CSG's Chomsky type 1 languages The following group can have equivalent computational power context free languages, CFL's context free grammars, CFG's NPDA nondeterministic pushdown automata Chomsky type 2 languages The following group can have equivalent computational power regular expressions regular languages regular grammars deterministic and nondeterministic finite automata, DFA, NFA, NFA epsilon Chomsky type 3 languages
Contents
Symbol Alphabet String or Word Language, Formal Language Regular Language Grammar, Formal Grammar Regular Grammar Context Free Language, CFL Greibach Normal Form, GNF Chomsky Normal Form CYK Algorithm Recursive Languages, Sets Recursively Enumerable Languages, Sets Chomsky Hierarchy of Grammars, Languages P and NP Classes of Languages Other links
Symbol
A character, glyph, mark.
An abstract entity that has no meaning by itself, often called uninterpreted. Letters from various alphabets, digits and special characters are the most commonly used symbols.
Alphabet
A finite set of symbols. An alphabet is often denoted by sigma, yet can be given any name. B = {0, 1} Says B is an alphabet of two symbols, 0 and 1. C = {a, b, c} Says C is an alphabet of three symbols, a, b and c. Sometimes space and comma are in an alphabet while other times they are meta symbols used for descriptions.
The building blocks of regular languages are symbols, concatenation of symbols to make strings (words), set union of strings and Kleene closure (denoted as *, also called the Kleene star, it should be typed as a superscript but this is plain text.) Informally, we use a syntax for regular expressions. using sigma as the set {0, 1} (an alphabet of two symbols) 01110 is a string starting with the symbol 0 and then concatenating 1, then 1, then 1, and finally concatenating 0. No punctuation is used between symbols or strings that are concatenated. (01+10) is a union of the two strings 01 and 10. The set {01, 10} (00+11)* is the Kleene closure of the union of 0 concatenated with 0 and 1 concatenated with 1. The Kleene closure (star) is defined as the concatenation of none, one, two, or any countable number strings it applies to. Examples of Kleene star: 1* is the set of strings {epsilon, 1, 11, 111, 1111, 11111, etc. } This set is infinite. (1100)* is the set of strings {epsilon, 1100, 11001100, 110011001100, etc. } (00+11)* is the set of strings {epsilon, 00, 11, 0000, 0011, 1100, 1111, 000000, 000011, 001100, etc. } note how the union ( + symbol) allows all possible choices of ordering when used with the Kleene star. (0+1)* is all possible strings of zeros and ones, often written as sigma * where sigma = {0, 1} (0+1)* (00+11) is all strings of zeros and ones that end with either 00 or 11. Note that concatenation does not have an operator symbol. (w)+ is a shorthand for (w)(w)* w is any string or expression and the superscript plus, + , means one or more copies of w are in the set defined by this expression. Formally, a regular language is defined on the bottom of page 28. Some versions of grep implement the regular expression search by simulating a Finite Automata. Note that grep uses a different syntax and uses a subset of ASCII characters for symbols.
V T P S
is is is is
a set of symbols called variables, typically S, A, B, ... a set of symbols called terminal, typically 0, 1, a, b, ... a set of productions the starting or goal variable from V
The productions P are of the form: A -> w Where A is a variable w is any concatenation of variables and terminals An example grammar is G = (V, T, P , S) where 0, 1 } and the productions, P , are: S -> 0A | 0 A -> 10A This grammar corresponds to the regular expression turn corresponds to the deterministic finite automata shown in Figure DFA1 . 0(10)* which in V = { S, A } T = {
Regular Grammar
A V T P S grammar is defined as G = (V, T, P, S) where: is a set of symbols called variables, v1, v2, ... ,vn is a set of symbols called terminal, t1, t2, ,,, ,tm is a set of productions is the starting or goal variable from V
The productions P are of the form: A -> w A -> wB Where A and B are variables w is any combination terminals, may be empty string Any regular grammar can be converted to an equivalent DFA, NFA, regular language or regular expression.
A -> a alpha where A is a variable in V, a is a terminal in T and alpha is none, one or more variables in V.
Note: Type 0 grammars can have infinite loops in the parser for the grammar when a string not in the grammar is input to the parser.
Class: P: machines NP: machines co-NP: machines PSPACE: machines NPSPACE: machines
Brief description the set of languages accepted by deterministic Turing in polynomial time the set of languages accepted by nondeterministic Turing in polynomial time the set of languages rejected by nondeterministic Turing in polynomial time the set of languages accepted by deterministic Turing in polynomial space the set of languages accepted by nondeterministic Turing
in polynomial space LOGSPACE: the set of languages accepted by deterministic Turing machines in logarithmic space NLOGSPACE:the set of languages accepted by nondeterministic Turing machines in logarithmic space EXP: the set of languages accepted by deterministic Turing machines in t(n)=2^cn time NEXP: the set of languages accepted by nondeterministic Turing machines in t(n)=2^cn time PEXP: the set of languages accepted by deterministic Turing machines in t(n)=2^p(n) time NPEXP: the set of languages accepted by nondeterministic Turing machines in t(n)=2^p(n) time UP: the set of languages accepted by unambiguous nondeterministic Turing machines, that have at least one accepting computation on any input, in polynomial time PP: the set of languages accepted by probabilistic polynomial-time Turing machines (not proved random = pseudo random) BPP: the set of languages accepted by bounded probabilistic polynomial-time Turing machines (balanced probability) RP: the set of languages accepted by random probabilistic polynomial-time Turing machines (one sided probability) co-RP: the set of languages accepted by RP machines with accept and reject probabilities reversed ZPP: RP intersection co-RP, the set of languages accepted by zero probability of error polynomial-time Turing machines
Function Classes
PF: machines NPF: machines the set of functions computed by deterministic Turing in polynomial time the set of functions computed by nondeterministic Turing in polynomial time
the set of functions computed by deterministic Turing in polynomial space the set of functions that enumerate the number of accepting computations of polynomial-time nondeterministic Turing
Hierarchies
PH: BH: QH: the union of classes known as the polynomial-time hierarchy the union of classes known as the Boolean hierarchy the union of classes known as the query hierarchy
Notes
An alphabet is a finite set of symbols. A string is a finite concatenation of symbols from an alphabet A language is a set of strings. A class is a set of languages. A hierarchy is a set of classes with relationships. "polynomial time" means there is a fixed polynomial p(n) and for each string x, accepted by a machine, the number of steps the machine takes is bounded by p(n). The "n" is the length of an input string x denoted |x|. In general the time bound is expressed an t(f(n)) as in EXP where t(n)=2^cn . "logarithmetic space" means there is a fixed f(n) and for each string x, accepted by a machine, the number of temporary storage places for symbols is bounded by f(n)= c*log(n). Let L be a language defined as a set L that is a subset of sigma star, for some finite alphabet sigma. L is the set of strings accepted by some restriction of a Turing machine. P is a set of languages where the restriction is a deterministic Turing machine with moves bounded by a polynomial in the length of x, |x| P = {L | language L is accepted by a deterministic Turing machine in polynomial time} co-P = {L | complement of L is in P} Note: P is closed under complementation but if NP = co-NP then PH collapses. #P = {f | f(x) = number of accepting paths in M(x) for some NP machine M} L in BPP implies there exists a set A in P, such that x in L implies Prob(y)[(x,y) in A] >= 1/2 and x not in L implies Prob(y)[(x,y) not in A] >= 1/2 L in RP implies there exists a set A in P, such that x in L implies Prob(y)[(x,y) in A] >= 1/2 and x not in L implies Prob(y)[(x,y) not in A] = 1 (no error on rejection) L in co-RP implies there exists a set A in P, such that x in L implies Prob(y)[(x,y) in A] = 1 and
x not in L implies Prob(y)[(x,y) not in A] >= 1/2 (no error on accepting) Where y is chosen randomly over {0,1}^q for some q bounded by a polynomial in |x| and A is a set of ordered pairs, x from L and y from {0,1}^q. Probabilities greater than be arbitrarily close to 1.0 using a polynomial number of tries. 1/2 + 1/( O(2^n) ) can be amplified to