Greedyalgorithm

Greedyalgorithm

• 2.
  Contents  What isgreedy algorithm?  Elements of greedy algorithm.  Application of greedy algorithm  TSP  Huffman Coding
• 3.
  What is GreedyAlgorithm?  In the hard words: A greedy algorithm is an algorithm that follows the problem solving heuristics of making the locally optimal choice at each stagewith the hope of finding a global optimum. (src: http://en.wikipedia.org/wiki/Greedy_algorithm)  Simplify: Choose the best choice that ‘reachable’ at current state
• 4.
  Elements of GreedyAlgorithm • The following are the two key ingredients to prove the Greedy Algorithms are optimal. 1. Greedy Choice Property 2. Optimal Substructure
• 5.
  Greedy Choice Property •Global optimal solution will be made with local optimal (greedy) choices. • In a greedy algorithm, the best choice will be selected at the moment to solve the sub problem that remains. • The choice of the greedy may depend on the choices so far, but it can’t depend on the future choices or on the solutions to the sub-problems.
• 6.
  Optimal Substructure • Optimalsubstructure is a property of Greedy solution. • A problem exhibits optimal substructure if an optimal solution to the problem contains within it optimal solutions to the sub- problems.
• 7.
  Procedure to provethe Optimal Greedy Algorithms 1. Consider globally-optimal solution 2. Show greedy choice at first step reduces problem to the same but smaller problem. Greedy Choice must be part of an optimal solution and made first. 3. Use induction to show greedy choice is best at each step i.e., optimal substructure
• 8.
  8 Applications of theGreedy Strategy  Optimal solutions:  change making for “normal” coin denominations  minimum spanning tree (MST)  single-source shortest paths  scheduling problems  Huffman codes  Approximations:  Traveling salesman problem (TSP)  knapsack problem  other combinatorial optimization problems
• 9.
  Sample Usage ofGreedy  For better explanation we use old simple problem: Travelling Salesman Problem:
• 10.
  TSP  The Problemis how to travel from city A and visit all city on the map, then back to city A again.  The rules: you only visit each city once and you can’t pass through any traversed path.
• 11.
  Solution:  Find theshortest path from city A(start) to any other city.  Because the nearest city is B, so we go to B BA
• 12.
   From B,we find any other city but A(because A has been visited) that has nearest path. So we choose C: Keep tuning on… BA C
• 13.
   From C,we look to nearest city again, but don’t look for A and B, because both has been visited. So we choose D. Soon end… BA C D
• 14.
   At thisnode(D), we can’t go to any city, because all neighbor of D has been visited. We go back to first city(A).  And that was how to solve TSP problem. BA C D
• 16.
  Huffman Coding Huffmancodes can be used to compress information  Like WinZip – although WinZip doesn’t use the Huffman algorithm  JPEGs do use Huffman as part of their compression process  The basic idea is that instead of storing each character in a file as an 8-bit ASCII value, we will instead store the more frequently occurring characters using fewer bits and less frequently occurring characters using more bits  On average this should decrease the filesize (usually ½)
• 17.
  Huffman Coding  Asan example, lets take the string: “duke blue devils”  We first to a frequency count of the characters:  e:3, d:2, u:2, l:2, space:2, k:1, b:1, v:1, i:1, s:1  Next we use a Greedy algorithm to build up a Huffman Tree  We start with nodes for each character e,3 d,2 u,2 l,2 sp,2 k,1 b,1 v,1 i,1 s,1
• 18.
  Huffman Coding  Wethen pick the nodes with the smallest frequency and combine them together to form a new node  The selection of these nodes is the Greedy part  The two selected nodes are removed from the set, but replace by the combined node  This continues until we have only 1 node left in the set
• 19.
  Huffman Coding e,3 d,2u,2 l,2 sp,2 k,1 b,1 v,1 i,1 s,1
• 20.
  Huffman Coding e,3 d,2u,2 l,2 sp,2 k,1 b,1 v,1 i,1 s,1 2
• 21.
  Huffman Coding e,3 d,2u,2 l,2 sp,2 k,1 b,1 v,1 i,1 s,1 22
• 22.
  Huffman Coding e,3 d,2u,2 l,2 sp,2 k,1 i,1 s,1 2 b,1 v,1 2 3
• 23.
  Huffman Coding e,3 d,2u,2 l,2 sp,2 k,1 i,1 s,1 2 b,1 v,1 2 34
• 24.
  Huffman Coding e,3 d,2 u,2l,2 sp,2 k,1 i,1 s,1 2 b,1 v,1 2 344
• 25.
  Huffman Coding e,3 d,2 u,2l,2 sp,2 k,1i,1 s,1 2 b,1 v,1 2 3 44 5
• 26.
  Huffman Coding e,3 d,2 u,2 l,2sp,2 k,1i,1 s,1 2 b,1 v,1 2 3 4 4 57
• 27.
  Huffman Coding e,3 d,2 u,2l,2 sp,2 k,1i,1 s,1 2 b,1 v,1 2 3 44 5 7 9
• 28.
  Huffman Coding e,3 d,2 u,2l,2 sp,2 k,1i,1 s,1 2 b,1 v,1 2 3 44 5 7 9 16
• 29.
  Huffman Coding  Nowwe assign codes to the tree by placing a 0 on every left branch and a 1 on every right branch  A traversal of the tree from root to leaf give the Huffman code for that particular leaf character  Note that no code is the prefix of another code
• 30.
  Huffman Coding e,3 d,2 u,2l,2 sp,2 k,1i,1 s,1 2 b,1 v,1 2 3 44 5 7 9 16 e 00 d 010 u 011 l 100 sp 101 i 1100 s 1101 k 1110 b 11110 v 11111
• 31.
  Huffman Coding  Thesecodes are then used to encode the string  Thus, “duke blue devils” turns into: 010 011 1110 00 101 11110 100 011 00 101 010 00 11111 1100 100 1101  When grouped into 8-bit bytes: 01001111 10001011 11101000 11001010 10001111 11100100 1101xxxx  Thus it takes 7 bytes of space compared to 16 characters * 1 byte/char = 16 bytes uncompressed
• 32.
  Advantage of Greedy Greedy is easy to be implemented. Just search the best choice from the current state that ‘reachable’ (has any paths or any connections).  In simple case, greedy often give you the best solution.
• 33.
  Drawback of Greedy In large and complex case, greedy doesn’t always give you the best solution, because it’s just search and take the best choice that you can reach from the current state.  It takes longer time than any other algorithms for big case of problem
• 34.
  Comp 122, Fall2003 34 Other greedy algorithms  Dijkstra’s algorithm for finding the shortest path in a graph » Always takes the shortest edge connecting a known node to an unknown node  Kruskal’s algorithm for finding a minimum-cost spanning tree » Always tries the lowest-cost remaining edge  Prim’s algorithm for finding a minimum-cost spanning tree » Always takes the lowest-cost edge between nodes in the spanning tree and nodes not yet in the spanning tree
• 35.
  THANK YOU!!