8. b What is Big Data English(1) tài liệu, giáo án, bài giảng , luận văn, luận án, đồ án, bài tập lớn về tất cả các lĩnh...
Paul Zikopoulos Director - IM WW Technical Professionals, WW Competitive Database, WW Big Data Tiger Team What is Big Data? © 2009 IBM Corporation Agenda What is Big Data? What makes Big Data different? What can you with Big Data? Big Data use cases The IBM Big Data platform Getting started © 2012 IBM Corporation New IM Technology Trends Information Integration and Governance & Big Data Trusted Relevant Governed Transactional & Collaborative Applications Analyze Integrate Business Analytics Applications Content Big Data Manage Master Data Cubes Streams Data External Information Sources Data Warehouses Content Information Governance Streaming Information Govern Security & Privacy Quality Standards Lifecycle © 2012 IBM Corporation © 2012 IBM Corporation …by the end of 2011, this was about 30 billion and growing even faster In 2005 there were 1.3 billion RFID tags in circulation… An increasingly sensor-enabled and instrumented business environment generates HUGE volumes of data with MACHINE SPEED characteristics… BILLION lines of code EACH engine generating 10 TB every 30 minutes! © 2012 IBM Corporation 350B Transactions/Year Meter Reads every 15 120M – meter reads/month 3.65B – meter reads/day © 2012 IBM Corporation In August of 2010, Adam Savage, of “Myth Busters,” took a photo of his vehicle using his smartphone He then posted the photo to his Twitter account including the phrase “Off to work.” Since the photo was taken by his smartphone, the image contained metadata revealing the exact geographical location the photo was taken By simply taking and posting a photo, Savage revealed the exact location of his home, the vehicle he drives, and the time he leaves for work © 2012 IBM Corporation The Social Layer in a Instrumented Interconnected World 30 billion RFID 12+ TBs tags today (1.3B in 2005) devices sold annually 2+ billion 25+ TBs of log data every day 76 million smart meters in 2009… 200M by 2014 camera phones world wide 100s of millions of GPS enabled data every day ? TBs of of tweet data every day 4.6 billion people on the Web by end 2011 © 2012 IBM Corporation Twitter Tweets per Second Record Breakers of 2011 © 2012 IBM Corporation Can a Social Media Persona be Monetized? 10 © 2012 IBM Corporation Public wind data is available on 284km x 284 km grids (2.5o LAT/LONG) More data means more accurate and richer models (adding hundreds of variables) - Vestas wind library at 2.5 PB: to grow to over PB in the near-term - Granularity 27km x 27km grids: driving to 9x9, 3x3 to 10m x 10m simulations Reduced turbine placement identification from weeks to hours 20 20 Perspective: The Vestas Wind library, as HD TV would take 70 years ©to watch 2012 IBM Corporation Optimize building energy consumption with centralized monitoring and control of building monitoring system Automates preventive and corrective maintenance of building corrective systems Uses Streams, InfoSphere BigInsights and Cognos 21 21 - Log Analytics Energy Bill Forecasting Energy consumption optimization Detection of anomalous usage Presence-aware energy mgt Policy enforcement © 2012 IBM Corporation Supply Chain Recommendation for Natural Disasters Capture market data to calculate cost of stock outs (high volume) Capture weather sensor data, analyses hurricane predicted path 22 Estimate impact on inventories Compute shipping and logistics costs Make recommendations and notify DHTML Result rendering © 2012 IBM Corporation Correlate combined risk and impending weather threats to optimize inventory and determine supply chain recommendations Dynamically updated risk assessment for assets in projected path Real-time projections of hurricane path 23 © 2012 IBM Corporation Bigger and Bigger Volumes of Data Retailers collect click-stream data from Web site interactions and loyalty card-drive transaction data – This traditional POS information is used by retailer for shopping basket analysis, inventory replenishment, +++ – But data is being provided to suppliers for customer buying analysis Healthcare has traditionally been dominated by paper-based systems, but this information is getting digitized Science is increasingly dominated by big science initiatives – Large-scale experiments generate over 15 PB of data a year and can’t be stored within the data center; then sent to laboratories Financial services are seeing larger volumes through smaller trading sizes, increased market volatility, and technological improvements in automated and algorithmic trading Improved instrument and sensory technology – Large Synoptic Survey Telescope’s GPixel camera generates 6PB+ of image data per year or consider Oil and Gas industry 24 © 2012 IBM Corporation Monetizing Relationships, Not Just Transactions Calling Network Amy Bearn How valuable is Amy to my mobile phone network? How likely is she to switch carriers? How many other customers will follow Retailer 32, Married, mother of 3, Accountant Telco Score: 91 CPG Score: 76 Fashion Score: 88 Telco company Merged Network Social Network 25 Public Database How valuable is Amy to my retail sales? Who does she influence? What they spend? © 2012 IBM Corporation Watson’s advanced analytic capabilities can sort through the equivalent of 200 MILLION pages of data to uncover an answer in SECONDS 26 © 2012 IBM Corporation Why Didn’t We Use All of the Big Data Before? 27 © 2012 IBM Corporation 28 © 2012 IBM Corporation The IBM Big Data Platform 29 © 2012 IBM Corporation The IBM Big Data Platform InfoSphere BigInsights Hadoop-based low latency analytics for variety and volume Hadoop Information Integration Stream Computing InfoSphere Information Server InfoSphere Streams Low Latency Analytics for streaming data High volume data integration and transformation MPP Data Warehouse IBM InfoSphere Warehouse IBM Netezza High Capacity Appliance Large volume structured data analytics Queryable Archive Structured Data 30 IBM Netezza 1000 BI+Ad Hoc Analytics on Structured Data IBM Smart Analytics System IBM Informix Timeseries Time-structured analytics Operational Analytics on Structured Data © 2012 IBM Corporation What Does a Big Data Platform Do? Analyze a Variety of Information Novel analytics on a broad set of mixed information that could not be analyzed before Analyze Information in Motion Streaming data analysis Large volume data bursts and ad-hoc analysis Analyze Extreme Volumes of Information Cost-efficiently process and analyze PBs of information Manage & analyze high volumes of structured, relational data discover and Experiment Ad-hoc analytics, data discovery and experimentation Manage and Plan Enforce data structure, integrity and control to ensure consistency for repeatable queries 31 © 2012 IBM Corporation Big Data Enriches the Information Management Ecosystem Active Archive Cost Optimization Master Data Enrichment via Life Events, Hobbies, Roles, +++ Establishing Information as a Service Audit MapReduce Jobs and tasks OLTP Optimization (SAP, checkout, +++) 32 Who Ran What, Where, and When? Managing a Governance Initiative © 2012 IBM Corporation Getting Started Get educated – – – – Forum content Books Analyst papers IBM web site: http://www-01.ibm.com/software/data/bigdata/ Start Planning – Free IBM Workshop – – – – 33 Best practices Business uses Big Data Roadmap and capabilities gaps Business value assessment © 2012 IBM Corporation THINK 34 34 © 2012 IBM Corporation ...Agenda What is Big Data? What makes Big Data different? What can you with Big Data? Big Data use cases The IBM Big Data platform Getting started © 2012 IBM Corporation New IM Technology... 2012 IBM Corporation What is BIG DATA ? All kinds of data Large volumes Valuable insight, but difficult to extract Often extremely time sensitive 13 © 2012 IBM Corporation What makes big data. .. - Noise Tina Mu Monetizable Intent Jo Jobs Not Relevant - Noise 11 Location Wishful Thinking Relocation SPAMbots Monetizable Intent © 2012 IBM Corporation 1.8 ZB ZB ZB=1T GB 4Trillion 8GB iPods