tính toán song song thoại nam parallelprocessing 06 speedup sinhvienzone com

Si nh Vi en Zo ne C om Speedup SinhVienZone.com Thoai Nam https://fb.com/sinhvienzonevn  C ne Zo  nh Vi en  Speedup & Efficiency Amdahl’s Law Gustafson’s Law Sun & Ni’s Law Si  om Outline Khoa Khoa h SinhVienZone.com ọc Kỹ thuật Máy tính - ĐHBK TP.HCM https://fb.com/sinhvienzonevn Speedup: S = Time(the most efficient sequential algorithm) / Time(parallel algorithm) Efficiency: E=S/N with N is the number of processors nh Vi en Si  Zo ne C  om Speedup & Efficiency Khoa Khoa h SinhVienZone.com ọc Kỹ thuật Máy tính - ĐHBK TP.HCM https://fb.com/sinhvienzonevn The main objective is to produce the results as soon as possible C  om Amdahl’s Law – Fixed Problem Size (1) Implications nh Vi en  Zo ne – (ex) video compression, computer graphics, VLSI routing, etc  Si – Upper-bound is – Make Sequential bottleneck as small as possible – Optimize the common case Modified Amdahl’s law for fixed problem size including the overhead Khoa Khoa h SinhVienZone.com ọc Kỹ thuật Máy tính - ĐHBK TP.HCM https://fb.com/sinhvienzonevn Sequential Parallel C Sequential om Amdahl’s Law – Fixed Problem Size (2) Parallel Tp T(1) nh Vi en Zo ne Ts Sequential P0 P1 P2 P3 P P5 P6 P7 P8 P9 Si T(N) Ts=T(1)  Tp= (1-)T(1) T(N) = T(1)+ (1-)T(1)/N Khoa Khoa h SinhVienZone.com ọc Kỹ thuật Máy tính - ĐHBK TP.HCM https://fb.com/sinhvienzonevn Number of processors .C om Amdahl’s Law – Fixed Problem Size (3) nh Vi en Zo ne Time(1) Speedup  Time( N ) Si T (1) 1 Speedup    as N   (1   )T (1) (1   )  T (1)   N N Khoa Khoa h SinhVienZone.com ọc Kỹ thuật Máy tính - ĐHBK TP.HCM https://fb.com/sinhvienzonevn ne C The overhead includes parallelism and interaction overheads om Enhanced Amdahl’s Law Si nh Vi en Zo T (1) Speedup   as N   (1   )T (1) Toverhead T (1)   Toverhead  N T (1) Khoa Khoa h SinhVienZone.com ọc Kỹ thuật Máy tính - ĐHBK TP.HCM https://fb.com/sinhvienzonevn Gustafson’s Law – Fixed Time (1) User wants more accurate results within a time limit om  ne C – Execution time is fixed as system scales – (ex) FEM (Finite element method) for structural analysis, FDM (Finite difference method) for fluid dynamics  Easy to measure Architecture independent Easy to model with an analytical expression No additional experiment to measure the work The measure of work should scale linearly with sequential time complexity of the algorithm nh Vi en – – – – – Zo Properties of a work metric Si  Time constrained seems to be most generally viable model! Khoa Khoa học Kỹ thuật Máy tính - ĐHBK TP.HCM SinhVienZone.com https://fb.com/sinhvienzonevn Ws W0 Si W(N) Sequential P0 nh Vi en Sequential Zo ne Parallel  = Ws / W(N) W(N) = W(N) + (1-)W(N)  W(1) = W(N) + (1-)W(N)N C P9 om Gustafson’s Law – Fixed Time (2) Sequential Khoa Khoa h SinhVienZone.com P0 P P2 P3 P4 P5 P6 P7 P8 P9 ọc Kỹ thuật Máy tính - ĐHBK TP.HCM https://fb.com/sinhvienzonevn om Gustafson’s Law – Fixed Time without overhead ne C Time = Work k W(N) = W Si nh Vi en Zo T (1) W (1).k W  (1    NW Speedup       (1   ) N T ( N ) W ( N ).k W Khoa Khoa h SinhVienZone.com ọc Kỹ thuật Máy tính - ĐHBK TP.HCM https://fb.com/sinhvienzonevn om Gustafson’s Law – Fixed Time with overhead ne Zo T (1) W (1).k W  (1    NW   (1    N    W0 T ( N ) W ( N ).k W  W0 1 W Si nh Vi en Speedup  C W(N) = W + W0 Khoa Khoa h SinhVienZone.com ọc Kỹ thuật Máy tính - ĐHBK TP.HCM https://fb.com/sinhvienzonevn Scale the largest possible solution limited by the memory space Or, fix memory usage per processor Speedup, nh Vi en – Time(1)/Time(N) for scaled up problem is not appropriate – For simple profile, and G(N) is the increase of parallel workload as the memory capacity increases N times Si  Zo ne C  om Sun and Ni’s Law – Fixed Memory (1) Khoa Khoa h SinhVienZone.com ọc Kỹ thuật Máy tính - ĐHBK TP.HCM https://fb.com/sinhvienzonevn nh Vi en – the increased memory N*M – The scaled work: W=W+(1- )G(N)W Speedup MC  Si  Zo ne  W=W+(1- )W Let M be the memory capacity of a single node N nodes: C  om Sun and Ni’s Law – Fixed Memory (2) Khoa Khoa h SinhVienZone.com   (1   )G ( N ) G( N )   (1   ) N ọc Kỹ thuật Máy tính - ĐHBK TP.HCM https://fb.com/sinhvienzonevn  om Sun and Ni’s Law – Fixed Memory (3) Definition: nh Vi en Theorem: If W = g (M ) for some homomorphism function g , g (cx)  g (c) g ( x) , then, with all data being shared by all available processors, the simplified memory-bounced speedup is W1  g ( N )WN   (1   )G ( N ) * SN   g (N ) G( N ) W1  WN   (1   ) N N Si  Zo ne C A function g is homomorphism if there exists a function g such that for any real number c and variable x, g (cx)  g (c) g ( x) Khoa Khoa h SinhVienZone.com ọc Kỹ thuật Máy tính - ĐHBK TP.HCM https://fb.com/sinhvienzonevn om Sun and Ni’s Law – Fixed Memory (4) Proof: * * W  W W1  g ( N )WN * N SN   * WN W  g ( N ) W * W1  N N N Si nh Vi en Zo ne C Let the memory requirement of Wn be M, Wn = g (M ) M is the memory requirement when node is available With N nodes available, the memory capacity will increase to N*M Using all of the available memory, for the scaled parallel * * W W portion N : N  g ( NM )  g ( N ) g (M )  g ( N )WN Khoa Khoa h SinhVienZone.com ọc Kỹ thuật Máy tính - ĐHBK TP.HCM https://fb.com/sinhvienzonevn om Speedup W1  G ( N )WN S  G( N ) W1  WN N Zo ne C * N Si nh Vi en – When the problem size is independent of the system, the problem size is fixed, G(N)=1  Amdahl’s Law – When memory is increased N times, the workload also increases N times, G(N)=N  Gustafson’s Law – For most of the scientific and engineering applications, the computation requirement increases faster than the memory requirement, G(N)>N Khoa Khoa h SinhVienZone.com ọc Kỹ thuật Máy tính - ĐHBK TP.HCM https://fb.com/sinhvienzonevn om Examples C 10 ne 10 Si nh Vi en S(Linear) S(Normal) Zo Speedup Processors Khoa Khoa h SinhVienZone.com ọc Kỹ thuật Máy tính - ĐHBK TP.HCM https://fb.com/sinhvienzonevn Parallelizing a code does not always result in a speedup; sometimes it actually slows the code down! This can be due to a poor choice of algorithm or to poor coding The best possible speedup is linear, i.e it is proportional to the number of processors: T(N) = T(1)/N where N = number of processors, T(1) = time for serial run A code that continues to speed up reasonably close to linearly as the number of processors increases is said to be scalable Many codes scale up to some number of processors but adding more processors then brings no improvement Very few, if any, codes are indefinitely scalable Si  nh Vi en Zo  ne C  om Scalability Khoa Khoa h SinhVienZone.com ọc Kỹ thuật Máy tính - ĐHBK TP.HCM https://fb.com/sinhvienzonevn  om Factors That Limit Speedup Software overhead  Load balancing Communication overhead Si  nh Vi en Zo ne C Even with a completely equivalent algorithm, software overhead arises in the concurrent implementation (e.g there may be additional index calculations necessitated by the manner in which data are "split up" among processors.) i.e there is generally more lines of code to be executed in the parallel program than the sequential program Khoa Khoa h SinhVienZone.com ọc Kỹ thuật Máy tính - ĐHBK TP.HCM https://fb.com/sinhvienzonevn ... Vi en  Speedup & Efficiency Amdahl’s Law Gustafson’s Law Sun & Ni’s Law Si  om Outline Khoa Khoa h SinhVienZone. com ọc Kỹ thuật Máy tính - ĐHBK TP.HCM https://fb .com/ sinhvienzonevn Speedup: ... S(Linear) S(Normal) Zo Speedup Processors Khoa Khoa h SinhVienZone. com ọc Kỹ thuật Máy tính - ĐHBK TP.HCM https://fb .com/ sinhvienzonevn Parallelizing a code does not always result in a speedup; sometimes... Scalability Khoa Khoa h SinhVienZone. com ọc Kỹ thuật Máy tính - ĐHBK TP.HCM https://fb .com/ sinhvienzonevn  om Factors That Limit Speedup Software overhead  Load balancing Communication overhead