Skip to main content
UploadLanguage (EN)Support
BusinessMobileSocial MediaMarketingTechnologyArt & PhotosCareerDesignEducationPresentations & Public SpeakingGovernment & NonprofitHealthcareInternetLawLeadership & ManagementAutomotiveEngineeringSoftwareRecruiting & HRRetailSalesServicesScienceSmall Business & EntrepreneurshipFoodEnvironmentEconomy & FinanceData & AnalyticsInvestor RelationsSportsSpiritualNews & PoliticsTravelSelf ImprovementReal EstateEntertainment & HumorHealth & MedicineDevices & HardwareLifestyle
Doug Sillars, profile picture
Uploaded byDoug Sillars
PPTX, PDF45 views

Ai powered images-zurichpydata

Doug Sillars gave a presentation on using AI to optimize images for the web. He discussed how images dominate web content and explained techniques like cropping, blurring objects, and generating alt text using machine learning models. Sillars also demonstrated how to train custom models for tasks like detecting sunglasses and adding filters to photos. The presentation concluded by emphasizing how AI and ML can simplify and automate image preparation and processing for digital content.

Technology◦
Download to read offline
Using AI to Power Your Images Doug Sillars Zurich PyData December 11, 2019
Contact Me: @DougSillars Doug.Sillars@gmail.com www.dougsillars.com Doug Sillars Freelance Developer Relations Performance Audits: Web/Native Workshops: Performance/Images/Video/AR/ML http://bit.ly/HighPerformanceAndroidApps
Images Dominate the Web HTTPArchive mobile websites 04/2019
Image Prep for the Web
Image Prep for the Web
Image Prep for the Web
Image Prep for the Web
Image Prep for the Web
Shortcuts for Image Prep?
Shortcuts for Image Prep? Ensure You are delivering the optimal image
Shortcuts: Taking screenshots of Photoshop
Shortcuts for Image Prep?
Images on the web named screen*shot 426,000 screenshots live on the web April 2019
Images on the web named screen*shot 106,000 sites with Screen Shots April 2019
Images on the web named screen*shot 5M sites tested 106,000 sites with Screen Shots
Images on the web named screen*shot 5M sites tested 106,000 sites with Screen Shots 2.1% of the web! =
Image Formats – Screenshots -> JPG https://dougsillars.com/2019/02/10/using-screenshots-in-production/ Sometimes our Shortcuts have lasting effects
Shortcuts for Image Prep?
Shortcuts for Image Prep?
Shortcuts for Image Prep? https://res.cloudinary.com/dougsillars/image/upload/c_crop,h_1000,w_1000,q_auto,f_auto/v1558503538/doug_headshot_alps _tfzado.jpg
Shortcuts for Image Prep? https://res.cloudinary.com/dougsillars/image/upload/c_crop,h_1000,w_1000,q_auto,f_auto/v1558503538/doug_headshot_alps _tfzado.jpg 120 KB
Shortcuts for Image Prep?
Shortcuts for Image Prep? https://res.cloudinary.com/dougsillars/image/upload/c_crop,h_1080,w_1000,q_auto,f_auto/ v1558504239/IMG_20160527_131042692_HDR_c21a3k.jpg
Shortcuts for Image Prep Manual crop
Shortcuts for Image Prep Manual crops are not a shortcut
Shortcuts for Image Prep: Detection of Objects https://res.cloudinary.com/dougsillars/image/upload/g_auto,c_crop,w_1000,h_1000/ v1558504239/IMG_20160527_131042692_HDR_c21a3k.jpg
Image Cropping in Production: Twitter
Image Cropping in Production: Twitter https://blog.twitter.com/engineering/en_us/topics/infrastructure/2018/Smart-Auto-Cropping-of-Images.html
Object Detection Using Machine Learning • ImageAI library • Tensorflow based • Need Image Model Library detector = ObjectDetection() detector.setModelTypeAsRetinaNet() detector.setModelPath( os.path.join(execution_path , "resnet50_coco_best_v2.0.1.h5")) detector.loadModel() detections = detector.detectObjectsFromImage(input_image=os.path.join(execution_path , "berwickontweed.jpg"), output_image_path=os.path.join(execution_path , "resnet.jpg")) resnet=plt.imread('resnet.jpg')
Object Detection Using Machine Learning
Object Detection Using Machine Learning
Object Detection with Machine Learning
Object Detection • Cropping
Object Detection • Cropping • Blurring objects
Object Detection • Cropping • Blurring objects • Object Removal http://bit.ly/bootsphoto
Object Detection • Cropping • Blurring objects • Object Removal • Alt Text Creation
Object Detection • Cropping • Blurring objects • Object Removal • Alt Text Creation Alt = “A Person and 2 dogs”
If the Object is not Modeled…
If the Object is not Modeled… •person : 99.72 •dog : 88.67 •dog : 96.92
…Try a new library • https://llama.vision
…Try a new library • https://llama.vision
Object Detection Using Machine Learning
ML ALT text in production alt="Image may contain: bridge, sky, tree, outdoor, water and nature"
ML ALT text in production alt="Image may contain: dog"
ML ALT text in production
…Or you can train a model
Google Object Detection
Google Object Detection: Model Creation
Google Object Detection: Model Creation
Google Object Detection: Model Creation
Google Object Detection: Model Creation
Train Your Model Precision: How often Model’s predictions were correct. In this case 1/6 is a false positive Recall: How often model found known objects. 1/6 is a missed prediction.
Deploy Model and Use
Deploy Model and Use
Model Results!!
Model Results!!
Image Fix 1: Cropping
Image Fix 1: Cropping
Image Fix 1: Cropping
Image Fix 2: Inpainting
Image Fix 2: Inpainting Create a mask of the area
Image Fix 2: Inpainting Create a mask of the area
Image Fix 2: Inpainting
Image Fix 2: Inpainting
InPaint: Works well on single color areas
InPaint: Works Not So well with lots of color
Tricking InPaint Remove “outlier” colors from the border Use the inside as inpainting area
Tricking InPaint Convert to Grayscale colors are 0 – 255 (1D) Kmeans to bucket the colors
Tricking InPaint
Tricking InPaint 10 most common gray colors found
Tricking InPaint 10 most common gray colors found Frequency
Tricking InPaint 1. Loop through Color list
Tricking InPaint 2. Ignore the White Center
Tricking InPaint 3. If Color is an outlier (lower than 25th percentile Higher than 75th) Mask and replace
Tricking InPaint
Tricking InPaint
Tricking InPaint
Sunglasses Project
Sunglasses Project
Manually edit 1,000s of glasses?
Sunglasses Project
Sunglasses Project
Sunglasses Project
Sunglasses Project 1. Remove Background
Sunglasses Project 1. Remove Background
Sunglasses Project 2. Make (slightly) transparent https://res.cloudinary.com/dougsillars/image /upload/e_make_transparent,o_70/v1558760 129/6658.png
Train a model!
Want to minimize frame capture
Train a model!
Beginning
Find the arms
Find the arms Finding an Odd number: Fail
Modify the border around the arms
Inpaint and upload
Putting it all together https://res.cloudinary.com/dougsillars/image/upload/v1558855691/IMG_20181207_163533_ibfm7a.jpg
Putting it all together https://res.cloudinary.com/dougsillars/image/upload/c_crop,g_face,w_1500,h_1500/v1558855691/IMG_2018120 7_163533_ibfm7a.jpg
Putting it all together https://res.cloudinary.com/dougsillars/image/upload/c_crop,g_face,w_1500,h_1500/v1558855691/IMG_2018120 7_163533_ibfm7a.jpg l_6558_f,o_80,e_make_transparent/
Putting it all together https://res.cloudinary.com/dougsillars/image/upload/c_crop,g_face,w_1500,h_1500/l_6558_f,o_80,e_make_transparent/ g_adv_eyes,fl_layer_apply/v1558855691/IMG_20181207_163533_ibfm7a.jpg
Demo Time https://res.cloudinary.com/dougsillars/image/upload/c_crop,g_face,w_1500,h_1500/l_6558_f,o_80,e_make_transparent/ g_adv_eyes,fl_layer_apply/v1558855691/IMG_20181207_163533_ibfm7a.jpg