Skip to main content
UploadLanguage (EN)Support
BusinessMobileSocial MediaMarketingTechnologyArt & PhotosCareerDesignEducationPresentations & Public SpeakingGovernment & NonprofitHealthcareInternetLawLeadership & ManagementAutomotiveEngineeringSoftwareRecruiting & HRRetailSalesServicesScienceSmall Business & EntrepreneurshipFoodEnvironmentEconomy & FinanceData & AnalyticsInvestor RelationsSportsSpiritualNews & PoliticsTravelSelf ImprovementReal EstateEntertainment & HumorHealth & MedicineDevices & HardwareLifestyle
IT
Uploaded byIdan Tohami
PPTX, PDF319 views

Building big data applications on AWS by Ran Tessler

Ran Tessler discusses building big data applications on AWS. He outlines architectural principles like using a decoupled data bus to store, process and retrieve data and answers. He demonstrates using AWS services like Kinesis Firehose for data collection, EMR with Spark and Hive for processing, and Redshift and QuickSight for analysis and visualization. Tessler references the Lambda architecture and its batch, speed and serving layers that can be implemented using AWS services like S3, DynamoDB, ElastiCache and Lambda.

Technology◦
Related topics:
Cloud Computing Insights•
Downloaded 10 times
Ran Tessler, Manager, Solutions Architecture Building Big Data Applications on AWS
A Modern Take on Alchemy Turning Data into Actionable Insights
What to Expect from this Session Big Data architectural principles Reference Lambda Architecture Live demo
Architectural Principles • Decoupled “data bus” Data → Store → Process → Answers • Use the right tool for the job Latency, throughput, access patterns • Apply Lambda architecture ideas Immutable (append-only) log, batch/speed/serving layer • Leverage AWS managed services No/low admin • Be cost conscious Big data ≠ big cost
Simplify Big Data Processing Ingest / collect store process / analyze consume / visualize Time to Answer (data freshness) Throughput
Demo http://aws.amazon.com/big-data/use-cases/
AccessLog - Common Log Format (CLF) 75.35.230.210 - - [20/Jul/2009:22:22:42 -0700] "GET /images/pigtrihawk.jpg HTTP/1.1" 200 29236
Did NASA’s STS-69 Mission Land … … On the right homepage?
Your First Big Data Application on AWS PROCESS STORE ANALYZE & VISUALIZE COLLECT
Your First Big Data Application on AWS PROCESS STORE COLLECT: Amazon Kinesis Firehose ANALYZE & VISUALIZE
Your First Big Data Application on AWS STORE COLLECT: Amazon Kinesis Firehose ANALYZE & VISUALIZE PROCESS: Amazon EMR with Spark & Hive
Your First Big Data Application on AWS PROCESS: Amazon EMR with Spark & Hive STORE ANALYZE & VISUALIZE: Amazon Redshift and Amazon QuickSight COLLECT: Amazon Kinesis Firehose
Reference Lambda Architecture
process store Apps Batch Layer Amazon Kinesis S3 Connector Amazon S3 Amazon Redshift Amazon EMR Presto Hive Pig Spark Lambda Architecture Serving Layer Amazon ElastiCache Amazon DynamoDB Amazon RDS Amazon ES Amazon Kinesis Speed Layer KCL AWS Lambda Spark Streaming Storm Amazon MLdata
Back to our demo… PROCESS: Amazon EMR with Spark & Hive STORE ANALYZE & VISUALIZE: Amazon Redshift and Amazon QuickSight COLLECT: Amazon Kinesis Firehose
DIY Download all steps: http://bit.ly/29fhcwu http://aws.amazon.com/big-data/use-cases/
tesslerr@amazon.com