IntroComputerVision23.pptx

IntroComputerVision23.pptx

• 1.
  Computer Vision CSE/ECE 576 LindaShapiro Professor of Computer Science & Engineering Professor of Electrical & Computer Engineering
• 2.
  Course Information • Time: –MW: 1:30-2:50 • Location: – ECE 037 • Contact: – shapiro@cs.uw.edu • TAs: – Kechun Liu – kechun@cs.washington.edu – Meredith Wu – wenjunw@uw.edu – Mehmet Saygin Seyfioğlu – msaygin@uw.edu • Website: – https://courses.cs.washington.edu/courses/cse576/23sp/ 2
• 3.
  Topics • Introduction • Colorand Texture • Image Coordinates, Transforms, and Resizing • Filters and Convolutions • Edges and Lines • Interest Operators, Image Matching, Image Stitching • Face Detection/Recognition • Machine Learning Overview including Neural Nets • Object Detection and Recognition with ML • Convolutional Neural Networks • CNN Applications • Motion/Optical Flow • Stereo and 3D Depth Perception 3
• 4.
  Grading (tentative) • Sixregular assignments (75%) • One Course Project (25%) • NO EXAMS (Yay!) 4
• 5.
  Assignments • Build avision library from the ground up • Mostly in C • Play with advanced tools, neural networks • Beginning: lots of skeleton code, explanations • End: less guidance, more experimentation 5
• 6.
  Assignment 1: Funwith Color 6
• 7.
  Assignment 2: ImageResizing and Filtering 7
• 8.
  Assignment 3: PanoramaStitching 8
• 9.
  Assignment 4: NeuralNetworks 9
• 10.
  Assignment 5: PyTorch 10
• 11.
  Assignment 6: OpticalFlow 11
• 12.
  Final Course Project:Machine Learning for Some Kind of Application 12 • a • c • b • d
• 13.
  Books 13 Older, but designed forundergrads and has the basics. Chapters available from our web page. Newest and available as a pdf online (both the 2010 and 2020 versions).
• 14.
  14
• 15.
  15
• 16.
  White 1m Shadow Car Horse Wheel Person Sky Road The car isin front of the pole 16
• 17.
  Computer Vision • LowLevel Vision – Measurements – Enhancements – Region segmentation – Features • Mid Level Vision – Reconstruction – Depth – Motion Estimation • High Level Vision – Category detection – Activity recognition – Deep understandings 18
• 18.
  Computer Vision • LowLevel Vision – Measurements – Enhancements – Region segmentation – Features • Mid Level Vision – Reconstruction – Depth – Motion Estimation • High Level Vision – Category detection – Activity recognition – Deep understandings 1m White Shadow 19
• 19.
  Measurement Brightness 20
• 20.
  Measurement Brightness Slide Credit: AlyoshaEfros 21 http://www.newworldencyclopedia.org/entry/Same_color_illusion
• 21.
  Measurement Length http://www.michaelbach.de/ot/sze_muelue/index.html Müller-Lyer Illusion Slide Credit:Alyosha Efros 22
• 22.
  Image Enhancement Image Inpainting,M. Bertalmío et al. http://www.iua.upf.es/~mbertalmio//restoration.html 23
• 23.
  Image Enhancement Image Inpainting,M. Bertalmío et al. http://www.iua.upf.es/~mbertalmio//restoration.html 24
• 24.
  Image Enhancement Image Inpainting,M. Bertalmío et al. http://www.iua.upf.es/~mbertalmio//restoration.html 25
• 25.
  Seam Carving [Shai &Avidan, SIGGRAPH 2007] 26 less important
• 26.
  Content-aware resizing usesimportant areas. Extends in horizontal direction and reduces in vertical. Traditional resizing uses and stretches the whole image. [Shai & Avidan, SIGGRAPH 2007] 27
• 27.
  Computer Vision • LowLevel Vision – Measurements – Enhancements – Region segmentation – Features • Mid Level Vision – Reconstruction – Depth – Motion Estimation • High Level Vision – Category detection – Activity recognition – Deep understandings The car is in front of the pole 28
• 28.
  Applications: 3D Scanning ScanningMichelangelo’s “The David” • The Digital Michelangelo Project - http://graphics.stanford.edu/projects/mich/ • UW Prof. Brian Curless, collaborator • 2 BILLION polygons, accuracy to .29mm 29
• 29.
  The Digital MichelangeloProject, Levoy et al. 30
• 30.
  31
• 31.
  32
• 32.
  33
• 33.
  34
• 34.
  35
• 35.
  Google’s 3D Maps Structureestimation from tourist photos 36
• 36.
  Apple’s 3D maps 37 https://www.youtube.com/watch?v=InIVv-LsgZE
• 37.
  Computer Vision • LowLevel Vision – Measurements – Enhancements – Region segmentation – Features • Mid Level Vision – Reconstruction – Depth – Motion Estimation • High Level Vision – Category detection – Activity recognition – Deep understandings – Pose estimation Car Horse Person Sky Road 38
• 38.
  Face detection • Manynew digital cameras now detect faces – Canon, Sony, Fuji, … Source: S. Seitz 39
• 39.
  Vision-based interaction: XboxKinect 40
• 40.
  How hard iscomputer vision? 41
• 41.
  Marvin Minsky, MIT Turingaward,1969 “In 1966, Minsky hired a first-year undergraduate student and assigned him a problem to solve over the summer: connect a television camera to a computer and get the machine to describe what it sees.” Crevier 1993, pg. 88
• 42.
  43
• 43.
  Marvin Minsky, MIT Turingaward,1969 Gerald Sussman, MIT (the undergraduate) “You’ll notice that Sussman never worked in vision again!” – Berthold Horn
• 44.
  Why vision isso hard? 45
• 45.
  Why is visionso hard? • Ill-posed problem [Sinha and Adelson 1993] 46
• 46.
  Challenges 1: viewpoint variation Michelangelo 1475-1564 slide by Fei Fei, Fergus & Torralba 47
• 47.
  Challenges 2: illumination slidecredit: S. Ullman 48
• 48.
  Challenges 3: occlusion Magritte, 1957slide by Fei Fei, Fergus & Torralba 49
• 49.
  Challenges 4: scale slideby Fei Fei, Fergus & Torralba 50
• 50.
  Challenges 5: deformation Xu,Beihong 1943 slide by Fei Fei, Fergus & Torralba 51
• 51.
  Challenges 6: backgroundclutter Klimt, 1913 slide by Fei Fei, Fergus & Torralba 52
• 52.
  Challenges 7: objectintra-class variation slide by Fei-Fei, Fergus & Torralba 53
• 53.
  Challenges 8: localambiguity slide by Fei-Fei, Fergus & Torralba 54
• 54.
  Challenges 9: theworld behind the image Slide Credit: Alyosha Efros 55
• 55.
  What Works Today? 56
• 56.
  • Reading licenseplates, zip codes, checks Svetlana Lazebnik 57
• 57.
  Biometrics Fingerprint scanners on manynew laptops, other devices Face recognition systems now beginning to appear more widely http://www.sensiblevision.com/ Source: S. Seitz 58
• 58.
  Mobile visual search:Google Goggles 59
• 59.
  Face detection • Manynew digital cameras now detect faces – Canon, Sony, Fuji, … Source: S. Seitz 60
• 60.
  Smile detection Sony Cyber-shot®T70 Digital Still Camera Source: S. Seitz 61
• 61.
  Face recognition: AppleiPhoto, Facebook, Google, etc 62
• 62.
  Object recognition (insupermarkets) LaneHawk by EvolutionRobotics “A smart camera is flush-mounted in the checkout lane, continuously watching for items. When an item is detected and recognized, the cashier verifies the quantity of items that were found under the basket, and continues to close the transaction. The item can remain under the basket, and with LaneHawk,you are assured to get paid for it… “ 63
• 63.
  Safety 64
• 64.
  Security 65
• 65.
  Automotive safety • Mobileye:Vision systems in high-end BMW, GM, Volvo models – Pedestrian collision warning – Forward collision warning – Lane departure warning – Headway monitoring and warning Source: A. Shashua, S. Seitz 66
• 66.
  Google cars Oct 9,2010. "Google Cars Drive Themselves, in Traffic". The New York Times. John Markoff June 24, 2011. "Nevada state law paves the way for driverless cars". Financial Post. Christine Dobby Aug 9, 2011, "Human error blamed after Google's driverless car sparks five-vehicle crash". The Star (Toronto) 67
• 67.
  Vision-based interaction: XboxKinect 68
• 68.
  Augmented reality, consumerproducts http://nconnex.com/wp/ 69
• 69.
  Special effects: shapeand motion capture Source: S. Seitz 70
• 70.
  Vision for robotics,space exploration Vision systems (JPL) used for several tasks • Panorama stitching • 3D terrain modeling • Obstacle detection, position tracking NASA'S Mars Exploration Rover Spirit captured this westward view from atop a low plateau where Spirit spent the closing months of 2007. Source: S. Seitz 71
• 71.
  Medical imaging Image guidedsurgery Grimson et al., MIT 3D imaging MRI, CT 72
• 72.
  73 Classification of 22q11.2DS •Treat 2D azimuth-elevation angle histogram as feature vector
• 73.
  Computer vision researchin healthcare assisted living, patient monitoring [Lan et al, PAMI 2012] autism screening http://www.gatech.edu/newsroom/release.h ml?nid=60509 74
• 74.
  Computer vision inthe real-world • Most examples are less than 7 years old • Very active research area. Many new applications to come. • A website of computer vision industries maintained by Prof. David Lowe (UBC) • Note: website is old but interesting • Note: David Lowe retired and moved to Google 2015 to 2018 http://www.cs.ubc.ca/~lowe/vision.html 75
• 75.
  Assignments • Assignment 1:Fun with Color • Assignment 2: Image Resizing and Filtering • Assignment 3: Panorama Stitching • Assignment 4: Neural Networks • Assignment 5: Pytorch • Assignment 6: Optical Flow • Course Project: Teams working on Machine Learning Projects 76
• 76.
  Assignment 1 • It’sabout color, which we will cover Wednesday. • It’s meant to be very easy, but you want to start it early. 77
• 77.
  Assignment 1 Parts •1. data structure for an image typedef struct{ int h, w, c; float *data; } image; • So an image is a 3D array with height, width and channels (like for colors). • data is floating point numbers between 0 and 1 78
• 78.
  Assignment 1: Parts Readthem • TODO #1: get_pixel and set_pixel • TODO #2: copy_image • TODO #3: rgb_to_grayscale • TODO #4: shift_image (shifts values) • TODO #5: clamp_image (get values between 0 and 1) • TODO #6: rgb_to_hsv • TODO #7: hsv_to_rgb 79
• 79.
  80 Have Fun