Stochastic Processes Markov Processes Derivation of Performance Measures Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Jane Hillston School of Informatics The University of Edinburgh Scotland 23rd September 2014 Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Assumptions Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Stochastic Process Formally, a stochastic model is one represented as a stochastic process; Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Stochastic Process Formally, a stochastic model is one represented as a stochastic process; A stochastic process is a set of random variables{X (t), t ∈ T }. Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Stochastic Process Formally, a stochastic model is one represented as a stochastic process; A stochastic process is a set of random variables{X (t), t ∈ T }. T is called the index set usually taken to represent time. Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Stochastic Process Formally, a stochastic model is one represented as a stochastic process; A stochastic process is a set of random variables{X (t), t ∈ T }. T is called the index set usually taken to represent time. Since we consider continuous time models T = R≥0 , the set of non-negative real numbers. Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures State Space The state space of a stochastic process is the set of all possible values that the random variables X (t) can assume. Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Assumptions Stochastic Processes Markov Processes Derivation of Performance Measures State Space The state space of a stochastic process is the set of all possible values that the random variables X (t) can assume. Each of these values is called a state of the process. Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Assumptions Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions State Space The state space of a stochastic process is the set of all possible values that the random variables X (t) can assume. Each of these values is called a state of the process. Any set of instances of {X (t), t ∈ T } can be regarded as a path of a particle moving randomly in the state space, S, its position at time t being X (t). Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions State Space The state space of a stochastic process is the set of all possible values that the random variables X (t) can assume. Each of these values is called a state of the process. Any set of instances of {X (t), t ∈ T } can be regarded as a path of a particle moving randomly in the state space, S, its position at time t being X (t). These paths are called sample paths or realisations of the stochastic process. Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Properties of Stochastic Processes In this course we will focus on stochastic processes with the following properties: Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Assumptions Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Properties of Stochastic Processes In this course we will focus on stochastic processes with the following properties: {X (t)} is a Markov process. This implies that {X (t)} has the Markov or memoryless property: given the value of X (t) at some time t ∈ T , the future path X (s) for s > t does not depend on knowledge of the past history X (u) for u < t, i.e. for t1 < · · · < tn < tn+1 , Pr(X (tn+1 ) = xn+1 | X (tn ) = xn , . . . , X (t1 ) = x1 ) = Pr(X (tn+1 ) = xn+1 | X (tn ) = xn ) Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Properties of Stochastic Processes In this course we will focus on stochastic processes with the following properties: {X (t)} is irreducible. This implies that all states in S can be reached from all other states, by following the transitions of the process. If we draw a directed graph of the state space with a node for each state and an arc for each event, or transition, then for any pair of nodes there is a path connecting them, i.e. the graph is strongly connected. Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Properties of Stochastic Processes In this course we will focus on stochastic processes with the following properties: {X (t)} is stationary: for any t1 , . . . tn ∈ T and t1 + τ, . . . , tn + τ ∈ T (n ≥ 1), then the process’s joint distributions are unaffected by the change in the time axis and so, FX (t1 +τ )...X (tn +τ ) = FX (t1 )...X (tn ) Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Properties of Stochastic Processes In this course we will focus on stochastic processes with the following properties: {X (t)} is time homogeneous: the behaviour of the system does not depend on when it is observed. In particular, the transition rates between states are independent of the time at which the transitions occur. Thus, for all t and s, it follows that Pr(X (t + τ ) = xk | X (t) = xj ) = Pr(X (s + τ ) = xk | X (s) = xj ). Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Exit rate and sojourn time In any stochastic process the time spent in a state is called the sojourn time. Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Assumptions Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Exit rate and sojourn time In any stochastic process the time spent in a state is called the sojourn time. In a Markov process the rate of leaving a state xi , qi the exit rate, is exponentially distributed with the rate which is the sum of all N X qi,j . the individual transitions that leave the state, i.e. qi = j=1,j6=i Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Exit rate and sojourn time In any stochastic process the time spent in a state is called the sojourn time. In a Markov process the rate of leaving a state xi , qi the exit rate, is exponentially distributed with the rate which is the sum of all N X qi,j . the individual transitions that leave the state, i.e. qi = j=1,j6=i This follows from the superposition principle of exponential distributions. Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Exit rate and sojourn time In any stochastic process the time spent in a state is called the sojourn time. In a Markov process the rate of leaving a state xi , qi the exit rate, is exponentially distributed with the rate which is the sum of all N X qi,j . the individual transitions that leave the state, i.e. qi = j=1,j6=i This follows from the superposition principle of exponential distributions. It follows that the sojourn time will be 1/qi . Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Exit rate and sojourn time In any stochastic process the time spent in a state is called the sojourn time. In a Markov process the rate of leaving a state xi , qi the exit rate, is exponentially distributed with the rate which is the sum of all N X qi,j . the individual transitions that leave the state, i.e. qi = j=1,j6=i This follows from the superposition principle of exponential distributions. It follows that the sojourn time will be 1/qi . Note: by the Markov property, the sojourn times are memoryless. Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Transition rates and transition probabilities At time τ , the probability that there is a state transition in the interval (τ, τ + dt) is qi dt + o(dt). Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Assumptions Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Transition rates and transition probabilities At time τ , the probability that there is a state transition in the interval (τ, τ + dt) is qi dt + o(dt). When a transition out of state xi occurs, the new state is xj with probability pij , which must depend only on i and j (Markov). Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Transition rates and transition probabilities At time τ , the probability that there is a state transition in the interval (τ, τ + dt) is qi dt + o(dt). When a transition out of state xi occurs, the new state is xj with probability pij , which must depend only on i and j (Markov). Thus, for i 6= j, i, j ∈ S, Pr(X (τ + dt) = j | X (τ ) = i) = qij dt + o(dt) where the qij = qi pij , by the decomposition property. Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Transition rates and transition probabilities At time τ , the probability that there is a state transition in the interval (τ, τ + dt) is qi dt + o(dt). When a transition out of state xi occurs, the new state is xj with probability pij , which must depend only on i and j (Markov). Thus, for i 6= j, i, j ∈ S, Pr(X (τ + dt) = j | X (τ ) = i) = qij dt + o(dt) where the qij = qi pij , by the decomposition property. The qij are called the instantaneous transition rates. Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Transition rates and transition probabilities At time τ , the probability that there is a state transition in the interval (τ, τ + dt) is qi dt + o(dt). When a transition out of state xi occurs, the new state is xj with probability pij , which must depend only on i and j (Markov). Thus, for i 6= j, i, j ∈ S, Pr(X (τ + dt) = j | X (τ ) = i) = qij dt + o(dt) where the qij = qi pij , by the decomposition property. The qij are called the instantaneous transition rates. The transition probability pij is the probability, given that a transition out of state i occurs, that it is the transition to state j. By the definition of conditional probability, this is pij = qij /qi . Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Infinitesimal Generator Matrix The state transition diagram of a Markov process captures all the information about the states of the system and the transitions which can occur between then. Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Infinitesimal Generator Matrix The state transition diagram of a Markov process captures all the information about the states of the system and the transitions which can occur between then. We can capture this information in a matrix, Q , termed the infinitesimal generator matrix. Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Infinitesimal Generator Matrix The state transition diagram of a Markov process captures all the information about the states of the system and the transitions which can occur between then. We can capture this information in a matrix, Q , termed the infinitesimal generator matrix. For a state space of size N, this is a N × N matrix, where entry q(i, j) or qi,j , records the transition rate of moving from state xi to state xj . Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Infinitesimal Generator Matrix The state transition diagram of a Markov process captures all the information about the states of the system and the transitions which can occur between then. We can capture this information in a matrix, Q , termed the infinitesimal generator matrix. For a state space of size N, this is a N × N matrix, where entry q(i, j) or qi,j , records the transition rate of moving from state xi to state xj . By convention, the diagonal entries qi,i are the negative row sum for row i, i.e. N X qi,i = − qi,j j=1,j6=i Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Steady state probability distribution In performance modelling we are often interested in the probability distribution of the random variable X (t) over the state space S, as the system settles into a regular pattern of behaviour. Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Steady state probability distribution In performance modelling we are often interested in the probability distribution of the random variable X (t) over the state space S, as the system settles into a regular pattern of behaviour. This is termed the steady state probability distribution. Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Steady state probability distribution In performance modelling we are often interested in the probability distribution of the random variable X (t) over the state space S, as the system settles into a regular pattern of behaviour. This is termed the steady state probability distribution. From this probability distribution we will derive performance measures based on subsets of states where some condition holds. Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Existence of a steady state probability distribution For every time-homogeneous, finite, irreducible Markov process with state space S, there exists a steady state probability distribution {πk , xk ∈ S} Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Assumptions Stochastic Processes Markov Processes Derivation of Performance Measures Existence of a steady state probability distribution For every time-homogeneous, finite, irreducible Markov process with state space S, there exists a steady state probability distribution {πk , xk ∈ S} This distribution is the same as the limiting or long term probability distribution: πk = lim Pr(X (t) = xk | X (0) = x0 ) t→∞ Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Assumptions Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Existence of a steady state probability distribution For every time-homogeneous, finite, irreducible Markov process with state space S, there exists a steady state probability distribution {πk , xk ∈ S} This distribution is the same as the limiting or long term probability distribution: πk = lim Pr(X (t) = xk | X (0) = x0 ) t→∞ This distribution is reached when the initial state no longer has any influence. Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Probability flux In steady state, πi is the proportion of time that the process spends in state xi . Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Assumptions Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Probability flux In steady state, πi is the proportion of time that the process spends in state xi . Recall qij is the instantaneous probability that the model makes a transition from state xi to state xj . Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Probability flux In steady state, πi is the proportion of time that the process spends in state xi . Recall qij is the instantaneous probability that the model makes a transition from state xi to state xj . Thus, in an instant of time, the probability that a transition will occur from state xi to state xj is the probability that the model was in state xi , πi , multiplied by the transition rate qij . Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Probability flux In steady state, πi is the proportion of time that the process spends in state xi . Recall qij is the instantaneous probability that the model makes a transition from state xi to state xj . Thus, in an instant of time, the probability that a transition will occur from state xi to state xj is the probability that the model was in state xi , πi , multiplied by the transition rate qij . This is called the probability flux from state xi to state xj . Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Global balance equations In steady state, equilibrium is maintained so for any state the total probability flux out is equal to the total probability flux into the state. πi × X qij = xj ∈S,j6=i | {z flux out of xi X (πj × qji ) xj ∈S,j6=i } | {z flux into xi Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes } Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Global balance equations In steady state, equilibrium is maintained so for any state the total probability flux out is equal to the total probability flux into the state. πi × X qij = xj ∈S,j6=i | {z flux out of xi X (πj × qji ) xj ∈S,j6=i } | {z flux into xi } (If this were not true the distribution over states would change. ) Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Global balance equations Recall that the diagonal elements of the infinitesimal generator matrix Q arePthe negative sum of the other elements in the row, i.e. qii = − xj ∈S,j6=i qij . Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Global balance equations Recall that the diagonal elements of the infinitesimal generator matrix Q arePthe negative sum of the other elements in the row, i.e. qii = − xj ∈S,j6=i qij . We can use this to rearrange the flux balance equation to be: X πj qji = 0. xj ∈S Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Global balance equations Recall that the diagonal elements of the infinitesimal generator matrix Q arePthe negative sum of the other elements in the row, i.e. qii = − xj ∈S,j6=i qij . We can use this to rearrange the flux balance equation to be: X πj qji = 0. xj ∈S Expressing the unknown values πi as a row vector π, we can write this as a matrix equation: πQ=0 Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Normalising constant The πi are unknown — they are the values we wish to find. Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Assumptions Stochastic Processes Markov Processes Derivation of Performance Measures Normalising constant The πi are unknown — they are the values we wish to find. If there are N states in the state space, the global balance equations give us N equations in N unknowns. Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Assumptions Stochastic Processes Markov Processes Derivation of Performance Measures Normalising constant The πi are unknown — they are the values we wish to find. If there are N states in the state space, the global balance equations give us N equations in N unknowns. However this collection of equations is irreducible. Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Assumptions Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Normalising constant The πi are unknown — they are the values we wish to find. If there are N states in the state space, the global balance equations give us N equations in N unknowns. However this collection of equations is irreducible. Fortunately, since {πi } is a probability distribution we also know that the normalisation condition holds: X πi = 1 xi ∈S Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Normalising constant The πi are unknown — they are the values we wish to find. If there are N states in the state space, the global balance equations give us N equations in N unknowns. However this collection of equations is irreducible. Fortunately, since {πi } is a probability distribution we also know that the normalisation condition holds: X πi = 1 xi ∈S With these n + 1 equations we can use standard linear algebra techniques to solve the equations and find the n unknowns, {πi }. Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Example Consider a system with multiple CPUs, each with its own private memory, and one common memory which can be accessed only by one processor at a time. Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Assumptions Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Example Consider a system with multiple CPUs, each with its own private memory, and one common memory which can be accessed only by one processor at a time. The CPUs execute in private memory for a random time before issuing a common memory access request. Assume that this random time is exponentially distributed with parameter λ (the average time a CPU spends executing in private memory between two common memory access requests is 1/λ). Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Example Consider a system with multiple CPUs, each with its own private memory, and one common memory which can be accessed only by one processor at a time. The CPUs execute in private memory for a random time before issuing a common memory access request. Assume that this random time is exponentially distributed with parameter λ (the average time a CPU spends executing in private memory between two common memory access requests is 1/λ). The common memory access duration is also assumed to be exponentially distributed, with parameter µ (the average duration of a common memory access is 1/µ). Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Example If the system has only one processor, it has only two states: 1 2 The processor is executing in its private memory; The processor is accessing common memory. Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Assumptions Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Example If the system has only one processor, it has only two states: 1 2 The processor is executing in its private memory; The processor is accessing common memory. The system behaviour can be modelled by a 2-state Markov process whose state transition diagram and generator matrix are as shown below: λ 1 µ 6 ? 2 Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Q= −λ λ µ −µ Stochastic Processes Example Markov Processes λ 1 µ 6 Derivation of Performance Measures ? 2 Q= −λ λ µ −µ Assumptions If we consider the probability flux in and out of state 1 we obtain: π1 λ = π2 µ. Similarly, for state 2: π2 µ = π1 λ. Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Example Markov Processes λ 1 µ 6 Derivation of Performance Measures ? 2 Q= −λ λ µ −µ Assumptions If we consider the probability flux in and out of state 1 we obtain: π1 λ = π2 µ. Similarly, for state 2: π2 µ = π1 λ. We know from the normalisation condition that: π1 + π2 = 1. Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Example Markov Processes λ 1 µ 6 Derivation of Performance Measures ? 2 Q= −λ λ µ −µ Assumptions If we consider the probability flux in and out of state 1 we obtain: π1 λ = π2 µ. Similarly, for state 2: π2 µ = π1 λ. We know from the normalisation condition that: π1 + π2 = 1. Thusthe steady stateprobability distribution is µ λ π= , . µ+λ µ+λ Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Example Markov Processes λ 1 µ 6 Derivation of Performance Measures ? 2 Q= −λ λ µ −µ Assumptions If we consider the probability flux in and out of state 1 we obtain: π1 λ = π2 µ. Similarly, for state 2: π2 µ = π1 λ. We know from the normalisation condition that: π1 + π2 = 1. Thusthe steady stateprobability distribution is µ λ π= , . µ+λ µ+λ From this we can deduce, for example, that the probability that the processor is executing in private memory is µ/(µ + λ). Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Solving the global balance equations In general our systems of equations will be too large to contemplate solving them by hand, so we want to be able to take advantage of linear algebra packages which can solve matrix equations of the form Ax = b, where A is an N × N matrix, x is a column vector of N unknowns, and b is a column vector of N values. Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Solving the global balance equations First we must resolve two problems: 1 Our global balance equation is expressed in terms of a row vector of unknowns π, π Q = 0: the unknowns. Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Assumptions Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Solving the global balance equations First we must resolve two problems: 1 Our global balance equation is expressed in terms of a row vector of unknowns π, π Q = 0: the unknowns. This problem is resolved by transposing the equation, i.e. QT π = 0, where the right hand side is now a column vector of zeros, rather than a row vector. Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Solving the global balance equations 2 We must eliminate the redundancy in the global balance equations and add in the normalisation condition. Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Assumptions Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Solving the global balance equations 2 We must eliminate the redundancy in the global balance equations and add in the normalisation condition. We replace one of the global balance equations by the normalisation condition. In QT this corresponds to replacing one row by a row of 1’s. We usually choose the last row and denote the modified matrix QT N. Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Solving the global balance equations 2 We must eliminate the redundancy in the global balance equations and add in the normalisation condition. We replace one of the global balance equations by the normalisation condition. In QT this corresponds to replacing one row by a row of 1’s. We usually choose the last row and denote the modified matrix QT N. We must also make the corresponding change to the “solution” vector 0, to be a column vector with 1 in the last row, and zeros everywhere else. We denote this vector, eN . Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Solving the global balance equations 2 We must eliminate the redundancy in the global balance equations and add in the normalisation condition. We replace one of the global balance equations by the normalisation condition. In QT this corresponds to replacing one row by a row of 1’s. We usually choose the last row and denote the modified matrix QT N. We must also make the corresponding change to the “solution” vector 0, to be a column vector with 1 in the last row, and zeros everywhere else. We denote this vector, eN . Now we can use any linear algebra solution package, such as MatLab to solve the resulting equation: QT N π = eN Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Example Consider the two-processor version of the multiprocessor with processors A and B. Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Assumptions Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Example Consider the two-processor version of the multiprocessor with processors A and B. We assume that the processors have different timing characteristics, the private memory access of A being governed by an exponential distribution with parameter λA , the common memory access of B being governed by an exponential distribution with parameter µB , etc. Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Example: state space Now the state space becomes: 1 A and B both executing in their private memories; 2 B executing in private memory, and A accessing common memory; 3 A executing in private memory, and B accessing common memory; 4 A accessing common memory, B waiting for common memory; 5 B accessing common memory, A waiting for common memory; Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Example: state space µ λB 2 1 3 λ i PP 1 µB A PP PP µB P P µA PP P λB λA PP P P - A 4 5 Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Assumptions Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Example: generator matrix Q= −(λA + λB ) λA λB 0 0 µA −(µA + λB ) 0 λB 0 µB 0 −(µB + λA ) 0 λA 0 0 µA −µA 0 0 µB 0 0 −µB Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Example: modified generator matrix QT N = −(λA + λB ) µA µB 0 0 λA −(µA + λB ) 0 0 µB λB 0 −(µB + λA ) µA 0 0 λB 0 −µA 0 1 1 1 1 1 Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Example: steady state probability distribution If we choose the following values for the parameters: λA = 0.05 λB := 0.1 µA = 0.02 µB = 0.05 solving the matrix equation, and rounding figures to 4 significant figures, we obtain: π = (0.0693, 0.0990, 0.1683, 0.4951, 0.1683) Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Deriving Performance Measures SYSTEM STATE TRANSITION DIAGRAM Q= ..... ..... ..... = EQUILIBRIUM PROBABILITY p , p , p , ..... DISTRIBUTION ..... , pN 1 Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes ..... ..... ..... ..... ..... MARKOV PROCESS 2 3 Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Deriving Performance Measures SYSTEM STATE TRANSITION DIAGRAM ..... ..... ..... ..... ..... MARKOV PROCESS Q= ..... ..... ..... = EQUILIBRIUM PROBABILITY p , p , p , ..... DISTRIBUTION ..... , pN 1 2 3 PERFORMANCE MEASURES e.g. throughput, response time, utilisation Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Deriving Performance Measures Broadly speaking, there are three ways in which performance measures can be derived from the steady state distribution of a Markov process. Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Assumptions Stochastic Processes Markov Processes Derivation of Performance Measures Deriving Performance Measures Broadly speaking, there are three ways in which performance measures can be derived from the steady state distribution of a Markov process. These different methods can be thought of as corresponding to different types of measure: state-based measures, e.g. utilisation; Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Assumptions Stochastic Processes Markov Processes Derivation of Performance Measures Deriving Performance Measures Broadly speaking, there are three ways in which performance measures can be derived from the steady state distribution of a Markov process. These different methods can be thought of as corresponding to different types of measure: state-based measures, e.g. utilisation; rate-based measures, e.g. throughput; Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Assumptions Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Deriving Performance Measures Broadly speaking, there are three ways in which performance measures can be derived from the steady state distribution of a Markov process. These different methods can be thought of as corresponding to different types of measure: state-based measures, e.g. utilisation; rate-based measures, e.g. throughput; other measures which fall outside the above categories, e.g. response time. Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions State-based measures State-based measures correspond to the probability that the model is in a state, or a subset of states, which satisfy some condition. Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions State-based measures State-based measures correspond to the probability that the model is in a state, or a subset of states, which satisfy some condition. For example, utilisation will correspond to those states where a resource is in use. Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions State-based measures State-based measures correspond to the probability that the model is in a state, or a subset of states, which satisfy some condition. For example, utilisation will correspond to those states where a resource is in use. If we consider the multiprocessor example, the utilisation of the common memory, Umem , is the total probability that the model is in one of the states in which the common memory is in use: Umem = π2 + π3 + π4 + π5 = 93.07% Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures State-based measures Other examples of state-based measures are idle time, or the number of jobs in a system. Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Assumptions Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions State-based measures Other examples of state-based measures are idle time, or the number of jobs in a system. Some measures such as the number of jobs will involve a weighted sum of steady state probabilities, weighted by the appropriate value (expectation). Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions State-based measures Other examples of state-based measures are idle time, or the number of jobs in a system. Some measures such as the number of jobs will involve a weighted sum of steady state probabilities, weighted by the appropriate value (expectation). For example, if we consider jobs waiting for the common memory to be queued in that subsystem, then the average number of jobs in the common memory, Nmem , is: Nmem = (1 × π2 ) + (1 × π3 ) + (2 × π4 ) + (2 × π5 ) = 1.594 Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Rate-based measures Rate-based measures are those which correspond to the predicted rate at which some event occurs. Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Rate-based measures Rate-based measures are those which correspond to the predicted rate at which some event occurs. This will be the product of the rate of the event, and the probability that the event is enabled, i.e. the probability of being in one of the states from which the event can occur. Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Example: rate-based measures In order to calculate the throughput of the common memory, we need the average number of accesses from either processor which it satisfies in unit time. Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Example: rate-based measures In order to calculate the throughput of the common memory, we need the average number of accesses from either processor which it satisfies in unit time. Xmem is thus calculated as: Xmem = (µA × (π2 + π4 )) + (µB × (π3 + π5 )) = 0.0287 or, approximately one access every 35 milliseconds. Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Other measures The other measures are those which are neither rate-based or state-based. Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Assumptions Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Other measures The other measures are those which are neither rate-based or state-based. In these cases, we usually use one of the operational laws to derive the information we need, based on values that we have obtained from solution of the model. Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Other measures The other measures are those which are neither rate-based or state-based. In these cases, we usually use one of the operational laws to derive the information we need, based on values that we have obtained from solution of the model. For example, applying Little’s Law to the common memory we see that Wmem = Nmem /Xmem = 1.594/0.0287 = 55.54 milliseconds Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Stochastic Hypothesis “The behaviour of a real system during a given period of time is characterised by the probability distributions of a stochastic process.” Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Assumptions Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Assumptions All delays and inter-event times are exponentially distributed. Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Assumptions All delays and inter-event times are exponentially distributed. (This will often not fit with observations of real systems.) Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Assumptions All delays and inter-event times are exponentially distributed. (This will often not fit with observations of real systems.) We make the assumption because of the nice mathematical properties of the exponential distribution, and because it is the only distribution giving us a Markov process. Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Assumptions All delays and inter-event times are exponentially distributed. (This will often not fit with observations of real systems.) We make the assumption because of the nice mathematical properties of the exponential distribution, and because it is the only distribution giving us a Markov process. Plus only a single parameter to be fitted (the rate), which can be easily derived from observations of the average duration. Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Assumptions All delays and inter-event times are exponentially distributed. (This will often not fit with observations of real systems.) We make the assumption because of the nice mathematical properties of the exponential distribution, and because it is the only distribution giving us a Markov process. Plus only a single parameter to be fitted (the rate), which can be easily derived from observations of the average duration. The Markov/memoryless assumption — future behaviour is only dependent on the current state, not on the past history — is a reasonable assumption for computer and communication systems, if we choose our states carefully. Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Assumptions All delays and inter-event times are exponentially distributed. (This will often not fit with observations of real systems.) We make the assumption because of the nice mathematical properties of the exponential distribution, and because it is the only distribution giving us a Markov process. Plus only a single parameter to be fitted (the rate), which can be easily derived from observations of the average duration. The Markov/memoryless assumption — future behaviour is only dependent on the current state, not on the past history — is a reasonable assumption for computer and communication systems, if we choose our states carefully. We generally assume that the Markov process is finite, time homogeneous and irreducible. Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes Stochastic Processes Markov Processes Derivation of Performance Measures Assumptions Exercise Consider the multiprocessor example, but with three processors, A, B and C sharing the common memory instead of two. List the states of the system, and draw the state transition diagram for this case. What is the difficulty in doing this and what further information do you need? Solution will be presented at the beginning of the next lecture. Jane Hillston School of Informatics The University of Edinburgh Scotland Performance Modelling — Lecture 3 Constructing and Solving Markov Processes

© Copyright 2017 ExploreDoc