Skip to main content
UploadLanguage (EN)Support
BusinessMobileSocial MediaMarketingTechnologyArt & PhotosCareerDesignEducationPresentations & Public SpeakingGovernment & NonprofitHealthcareInternetLawLeadership & ManagementAutomotiveEngineeringSoftwareRecruiting & HRRetailSalesServicesScienceSmall Business & EntrepreneurshipFoodEnvironmentEconomy & FinanceData & AnalyticsInvestor RelationsSportsSpiritualNews & PoliticsTravelSelf ImprovementReal EstateEntertainment & HumorHealth & MedicineDevices & HardwareLifestyle
Slideshare, profile picture
Uploaded bySlideshare
PPTX, PDF685 views

Instance based learning

Instance-based learning, also known as lazy learning, is a non-parametric learning method where the training data is stored and a new instance is classified based on its similarity to the nearest stored instances. It is similar to a desktop in that all data is kept in memory. The key aspects are setting the K value for the K-nearest neighbors algorithm and the distance metric such as Euclidean distance. Training involves storing all input data, finding the K nearest neighbors of each test instance, and classifying based on the majority class of those neighbors.

Education◦Technology◦
Download to read offline
Instance-Based Learning V.Saranya AP/CSE Sri Vidya College of Engineering and Technology, Virudhunagar
• Instance-Based Learning (Lazy Learning) –Learning = storing all “training” instances –Classification = an instance gets a classification equal to the classification of the nearest instances to the instance
Instance-Based Learning Its very similar to a Desktop!!
Instance/Memory-based Learning • Non-parameteric – Hypothesis(Assumption) complexity grows with the data • Memory-based learning – Construct hypotheses directly from the training data itself 4
5 K Nearest Neighbors • The key issues involved in training this model includes setting – the variable K • Validation techniques (ex. Cross validation) – the type of distant metric • Euclidean measure 2 1 )(),( D i YiXiYXDist
6 Figure K Nearest Neighbors Example X Stored training set patterns X input pattern for classification --- Euclidean distance measure to the nearest three patterns
7 Store all input data in the training set For each pattern in the test set Search for the K nearest patterns to the input pattern using a Euclidean distance measure For classification, compute the confidence for each class as Ci /K, (where Ci is the number of patterns among the K nearest patterns belonging to class i.) The classification for the input pattern is the class with the highest confidence.
8 Training parameters and typical settings • Number of nearest neighbors – The numbers of nearest neighbors (K) should be based on cross validation over a number of K setting. – When k=1 is a good baseline model to benchmark against. – A good rule-of-thumb numbers is k should be less than the square root of the total number of training patterns.
9 Training parameters and typical settings • Input compression – Since KNN is very storage intensive, we may want to compress data patterns as a preprocessing step before classification. – Using input compression will result in slightly worse performance. – Sometimes using compression will improve performance because it performs automatic normalization of the data which can equalize the effect of each input in the Euclidean distance measure.
Issues • Distance measure – Most common: Euclidean – Better distance measures: normalize each variable by standard deviation – For discrete data, can use hamming distance • Choosing k – Increasing k reduces variance, increases bias • For “high-dimensional space”, problem that the nearest neighbor may not be very close at all! • Memory-based technique. Must make a pass through the data for each classification. This can be prohibitive for large data sets. • Indexing the data can help; for example KD trees 10