Skip to main content
UploadLanguage (EN)Support
BusinessMobileSocial MediaMarketingTechnologyArt & PhotosCareerDesignEducationPresentations & Public SpeakingGovernment & NonprofitHealthcareInternetLawLeadership & ManagementAutomotiveEngineeringSoftwareRecruiting & HRRetailSalesServicesScienceSmall Business & EntrepreneurshipFoodEnvironmentEconomy & FinanceData & AnalyticsInvestor RelationsSportsSpiritualNews & PoliticsTravelSelf ImprovementReal EstateEntertainment & HumorHealth & MedicineDevices & HardwareLifestyle
Gabriel Peyré, profile picture
Uploaded byGabriel Peyré
PDF, PPTX1,050 views

Low Complexity Regularization of Inverse Problems - Course #1 Inverse Problems

This document provides an overview of an upcoming course on inverse problems and regularization. The course will cover three topics: inverse problems, compressed sensing, and sparsity and L1 regularization. Inverse problems involve recovering an unknown signal x0 from noisy observations. Regularization is used to incorporate prior information and make the problem well-posed. Compressed sensing allows signals to be sampled below the Nyquist rate if they are sparse. The L1 norm is used as a convex relaxation of the sparsity prior, allowing sparse recovery problems to be solved as convex programs.

Education◦Technology◦Health & Medicine◦
Related topics:
Image Processing Techniques•
Download as PDF, PPTX
Low Complexity Regularization of Inverse Problems Cours #1 Inverse Problems Gabriel Peyré www.numerical-tours.com
Overview of the Course • Course #1: Inverse Problems • Course #2: Recovery Guarantees • Course #3: Proximal Splitting Methods
Overview • Inverse Problems • Compressed Sensing • Sparsity and L1 Regularization
Inverse Problems Recovering x0 RN from noisy observations y = x 0 + w 2 RP
Inverse Problems Recovering x0 RN from noisy observations y = x 0 + w 2 RP Examples: Inpainting, super-resolution, . . . x0
Inverse Problems in Medical Imaging x = (p✓k )16k6K
Inverse Problems in Medical Imaging x = (p✓k )16k6K Magnetic resonance imaging (MRI): x ˆ ˆ x = (f (!))!2⌦
Inverse Problems in Medical Imaging x = (p✓k )16k6K Magnetic resonance imaging (MRI): x ˆ Other examples: MEG, EEG, . . . ˆ x = (f (!))!2⌦
Inverse Problem Regularization Observations: y = x0 + w 2 RP . Estimator: x(y) depends only on observations y parameter
Inverse Problem Regularization Observations: y = x0 + w 2 RP . Estimator: x(y) depends only on observations y parameter Example: variational methods 1 x(y) 2 argmin ||y x||2 + J(x) x2RN 2 Data fidelity Regularity
Inverse Problem Regularization Observations: y = x0 + w 2 RP . Estimator: x(y) depends only on observations y parameter Example: variational methods 1 x(y) 2 argmin ||y x||2 + J(x) x2RN 2 Data fidelity Regularity Choice of : tradeo Noise level ||w|| Regularity of x0 J(x0 )
Inverse Problem Regularization Observations: y = x0 + w 2 RP . Estimator: x(y) depends only on observations y parameter Example: variational methods 1 x(y) 2 argmin ||y x||2 + J(x) x2RN 2 Data fidelity Regularity Choice of : tradeo No noise: Noise level ||w|| 0+ , minimize Regularity of x0 J(x0 ) x? 2 argmin J(x) x2RQ , x=y
Inverse Problem Regularization Observations: y = x0 + w 2 RP . Estimator: x(y) depends only on observations y parameter Example: variational methods 1 x(y) 2 argmin ||y x||2 + J(x) x2RN 2 Data fidelity Regularity Choice of : tradeo No noise: This course: Noise level ||w|| 0+ , minimize Regularity of x0 J(x0 ) x? 2 argmin J(x) x2RQ , x=y Performance analysis. Fast computational scheme.
Overview • Inverse Problems • Compressed Sensing • Sparsity and L1 Regularization
Compressed Sensing [Rice Univ.] x0 ˜
Compressed Sensing [Rice Univ.] x0 ˜ y[i] = hx0 , 'i i P measures N micro-mirrors
Compressed Sensing [Rice Univ.] x0 ˜ y[i] = hx0 , 'i i P measures P/N = 1 N micro-mirrors P/N = 0.16 P/N = 0.02
CS Acquisition Model ˜ CS is about designing hardware: input signals f L2 (R2 ). Physical hardware resolution limit: target resolution f 2 x0 2 L ˜ array resolution CS hardware x0 2 R N micro mirrors RN . y 2 RP
CS Acquisition Model ˜ CS is about designing hardware: input signals f L2 (R2 ). Physical hardware resolution limit: target resolution f x0 2 L ˜ array resolution x0 2 R N micro mirrors y 2 RP CS hardware , , ... 2 RN . , Operator x0
Overview • Inverse Problems • Compressed Sensing • Sparsity and L1 Regularization
Redundant Dictionaries Dictionary =( m )m RQ N ,N Q. Q N
Redundant Dictionaries Dictionary Fourier: =( m m )m = ei RQ N ,N Q. ·, m frequency Q N
Redundant Dictionaries Dictionary Fourier: m )m =( m N ,N Q. =e i ·, m frequency Wavelets: m RQ = (2 j R x m = (j, , n) scale position orientation n) =1 Q N =2
Redundant Dictionaries Dictionary Fourier: m )m =( m N ,N Q. =e i ·, m frequency Wavelets: m RQ = (2 j R x m = (j, , n) scale position orientation n) DCT, Curvelets, bandlets, . . . =1 Q N =2
Redundant Dictionaries Dictionary Fourier: m )m =( m N ,N Q. =e i ·, m frequency Wavelets: m RQ = (2 j R x m = (j, , n) scale position orientation n) DCT, Curvelets, bandlets, . . . Synthesis: f = m xm m = =1 x. =f Q x N Coe cients x Image f = x =2
Sparse Priors Ideal sparsity: for most m, xm = 0. Coe cients x J0 (x) = # {m xm = 0} Image f0
Sparse Priors Ideal sparsity: for most m, xm = 0. Coe cients x J0 (x) = # {m xm = 0} Sparse approximation: f = x where argmin ||f0 x||2 + T 2 J0 (x) x2RN Image f0
Sparse Priors Coe cients x Ideal sparsity: for most m, xm = 0. J0 (x) = # {m xm = 0} Sparse approximation: f = x where argmin ||f0 x||2 + T 2 J0 (x) x2RN Orthogonal : = = IdN f0 , m if | f0 , xm = 0 otherwise. f= ST (f0 ) m | > T, ST Image f0
Sparse Priors Coe cients x Ideal sparsity: for most m, xm = 0. J0 (x) = # {m xm = 0} Sparse approximation: f = x where argmin ||f0 x||2 + T 2 J0 (x) x2RN Orthogonal : = = IdN f0 , m if | f0 , xm = 0 otherwise. f= ST (f0 ) Non-orthogonal : NP-hard. m | > T, ST Image f0
Convex Relaxation: L1 Prior J0 (x) = # {m xm = 0} Image with 2 pixels: x2 x1 q=0 J0 (x) = 0 J0 (x) = 1 J0 (x) = 2 null image. sparse image. non-sparse image.
Convex Relaxation: L1 Prior J0 (x) = # {m xm = 0} Image with 2 pixels: x2 J0 (x) = 0 J0 (x) = 1 J0 (x) = 2 null image. sparse image. non-sparse image. x1 q=0 q priors: q=1 q = 1/2 Jq (x) = m |xm |q q = 3/2 q=2 (convex for q 1)
Convex Relaxation: L1 Prior J0 (x) = # {m xm = 0} J0 (x) = 0 J0 (x) = 1 J0 (x) = 2 Image with 2 pixels: x2 null image. sparse image. non-sparse image. x1 q=0 q q=1 q = 1/2 Jq (x) = priors: m Sparse 1 prior: |xm |q J1 (x) = m |xm | q = 3/2 q=2 (convex for q 1)
L1 Regularization x0 RN coe cients
L1 Regularization x0 RN coe cients f0 = x0 RQ image
L1 Regularization x0 RN coe cients f0 = x0 RQ image K w y = Kf0 + w RP observations
L1 Regularization x0 RN coe cients f0 = x0 RQ image K w = K ⇥ ⇥ RP N y = Kf0 + w RP observations
L1 Regularization x0 RN coe cients f0 = x0 RQ image K y = Kf0 + w RP observations w = K ⇥ ⇥ RP Sparse recovery: f = N x where x solves 1 min ||y x||2 + ||x||1 x RN 2 Fidelity Regularization
Noiseless Sparse Regularization Noiseless measurements: x x= x argmin x=y m y |xm | y = x0
Noiseless Sparse Regularization Noiseless measurements: y = x0 x x= x argmin x=y m x x= y |xm | x argmin x=y m y |xm |2
Noiseless Sparse Regularization Noiseless measurements: y = x0 x x= x argmin x=y m x x= y |xm | x argmin x=y m y |xm |2 Convex linear program. Interior points, cf. [Chen, Donoho, Saunders] “basis pursuit”. Douglas-Rachford splitting, see [Combettes, Pesquet].
Noisy Sparse Regularization Noisy measurements: x y = x0 + w 1 argmin ||y x||2 + ||x||1 x RQ 2 Data fidelity Regularization
Noisy Sparse Regularization Noisy measurements: y = x0 + w x 1 argmin ||y x||2 + ||x||1 x RQ 2 Data fidelity Regularization x argmin ||x||1 || x y|| x Equivalence | x= y|
Noisy Sparse Regularization Noisy measurements: y = x0 + w x 1 argmin ||y x||2 + ||x||1 x RQ 2 Data fidelity Regularization x argmin ||x||1 || x y|| Algorithms: Iterative soft thresholding Forward-backward splitting see [Daubechies et al], [Pesquet et al], etc Nesterov multi-steps schemes. x Equivalence | x= y|
Inpainting Problem K Measures: y = Kf0 + w (Kf )i = ⇢ 0 if i 2 ⌦, fi if i 2 ⌦. /