Thuyết trình big data

Thuyết trình big data

Big Data GVGD: TS Nguyễn Đức Thái NHÓM Memory storage… Computer Memory: 640K Ought to be Enough for Anyone How much data?  billion people  Google processes 100 PB/day; million servers  Facebook has 300 PB + 500 TB/day; 35% of world’s photos  YouTube 1000 PB video storage; billion views/day  Twitter processes 124 billion tweets/year  SMS messages – 6.1T per year  US Cell Calls – 2.2T minutes per year  US Credit cards - 1.4B Cards; 20B transactions/year Contents Big Data Overview Big Data Technology Today SQL vs NoSQL Big Data Security Big data trends Demo with MongoDB & Ref docs Big Data Overview (tt) “Big data is not a single technology but a combination of old and new tech-nologies that helps companies gain actionable insight” (“Big Data For DummiesPublished by John Wiley & Sons, Inc ” book reference) Big Data Overview (tt) Characteristics of Big Data Sources of Big Data Social Media Website ERP Network Switches RFID Examining Big Data Types  Structured Data Structured Data(…) Computer- or machine-generated: Machine-generated data generally refers to data that is created by a machine without human intervention (Sensor data, Web log data, Point-ofsale data, Financial data…) Human-generated: This is data that humans, in interaction with computers, supply (Input data, Clickstream data, Gaming-related data…) 2.Big Data Technology Today(tt)  Open-source software framework from Apache Hadoop  Google MapReduce  GFS (Google File System)  HDFS  Map/Reduce SQL vs NoSQL File SQL DBMS Data storage NoSQL SQL vs NoSQL (…) A relational database is a set of tables containing data fitted into predefined categories Each table contains one or more data categories in columns Each row contains a unique instance of data for the categories defined by the columns SQL vs NoSQL (…)  Key-value stores As the name implies, a key-value store is a system that stores values indexed for retrieval by keys Some of the market leaders: Riak Amazon Dynamo Voldermort SQL vs NoSQL (…) Column-oriented databases columnoriented databases contain one extendable column of closely related data Some of the market leaders: HBase Cassandra SQL vs NoSQL (…) Document-based stores These databases store and organize data as collections of documents, rather than as structured tables with uniform sized fields for each record Some of the market leaders: MongoDB CouchDB SimpleDB SQL vs NoSQL (…) SQL 2008 Data storage capacity SQL vs NoSQL (…) GridFS stores files in two collections:  chunks stores the binary chunks For details, see The chunks Collection  files stores the file’s metadata For details, see The files Collection SQL vs NoSQL (…) The files Collection The chunks Collection BSON Types SQL vs NoSQL (…) Big Data Security • • • • • • Secure computations in distributed programming frameworks Security best practices for non-relational data stores Secure data storage and transactions logs Cryptographically enforced access control and secure communication Granular access control Real-time security/compliance monitoring Big Data Security (…) Technical Recommendations for sercurity • • • • • • • • Use Kerberos for node authentication Use file layer encryption Data anonymization Use key management Deployment validation Use secure communication Tokenization Cloud database controls Big data trends • Big data – of the people, by the • • • • • people, for the people Big data and social computing Cloud computing In memmory computing Mobile Applications and HTML5 Internet and big data Demo with MongoDB & Ref docs  Ref docs:  Judith Hurwitz, Alan Nugent, Dr Fern Halper, and Marcia Kaufman: Big Data For Dummies John Wiley & Sons, Inc 2013  “Technology Trends for 2013” prepared by Kaushal Amin, Chief Technology Officer, KMS Technology – Atlanta, GA, USA  Website: http://hadoop.apache.org/  Demo with MongoDB Thank You !