Big Data Strategies for Agile Business Framework, Practices, and Transformation Roadmap Big Data Strategies for Agile Business Framework, Practices, and Transformation Roadmap By Bhuvan Unhelkar, PhD, FACS CRC Press Taylor & Francis Group 6000 Broken Sound Parkway NW, Suite 300 Boca Raton, FL 33487-2742 © 2018 by Taylor & Francis Group, LLC CRC Press is an imprint of Taylor & Francis Group, an Informa business No claim to original U.S Government works Printed on acid-free paper International Standard Book Number-13: 978-1-498-72438-8 (Hardback) This book contains information obtained from authentic and highly regarded sources Reasonable efforts have been made to publish reliable data and information, but the author and publisher cannot assume responsibility for the validity of all materials or the consequences of their use The authors and publishers have attempted to trace the copyright holders of all material reproduced in this publication and apologize to copyright holders if permission to publish in this form has not been obtained If any copyright material has not been acknowledged, please write and let us know so we may rectify in any future reprint Except as permitted under U.S Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying, microfilming, and recording, or in any information storage or retrieval system, without written permission from the publishers For permission to photocopy or use material electronically from this work, please access www.copyright.com (http://www.copyright com/) or contact the Copyright Clearance Center, Inc (CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400 CCC is a not-for-profit organization that provides licenses and registration for a variety of users For organizations that have been granted a photocopy license by the CCC, a separate system of payment has been arranged Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe Visit the Taylor & Francis Web site at http://www.taylorandfrancis.com and the CRC Press Web site at http://www.crcpress.com Dedicated to these dear friends who departed (some before their time) in the span of a year as this book was being written May You All Rest in Peace! Padmanaabh Desai Ed Yourdon Houman Younessi Warren Irish Kamlesh Chaudhary Barry Gunn Dilip Thakar Arvind Swami Any sufficiently advanced technology is indistinguishable from magic. Arthur C Clarke Freedom from the desire for an answer is essential to the understanding of a problem. J Krishnamurti So it is incumbent on me to know myself, to know it completely, to know its minutiae, its characteristics, its subtleties, and its very atoms Kahlil Gibran Contents List of Figures .xxiii List of Tables xxix Foreword .xxxiii Preface xxxv Acknowledgments xlv About the Author xlvii Domain Terms and Acronyms xlix Section I INTRODUCTION TO BIG DATA STRATEGIES AND OUTLINE OF BIG DATA FRAMEWORK FOR AGILE BUSINESS (BDFAB) Introduction to BIG Data and Agile Business .3 Chapter Objectives Big Data and Business Value Data Value in Decisions Big Data Differentiator Business Agility as a Big Data Opportunity Data-Driven Decisions, Information, and Knowledge Strategic Approach to Big Data Setting the Scene for Strategies Understanding and Transcending Analytics and Technologies 12 Data Science to Business Leadership 17 Envisioning a Holistic Big Data Strategy .18 Big Data as Agile Business Enabler 23 Agile and Big Data 23 Types and Sizes of Organizations and Their Big Data Capabilities 24 Business Agility Is Fast and Accurate Decision Making with Finer Levels of Granularity 24 Composite Agile Method and Strategy 26 Lean, Agile, and Big Data 27 Big Data– Driven Business Strategies 28 External Growth of the Business 28 Internal Optimization of Business Processes and Structure 28 Risk Management and Compliance with Big Data 31 ix 490 ◾ Big Data Strategies for Agile Business ◾◾ SAS : http://www.sas.com/en_us/software/cloud-analytics.html Offers Cloud-based analytics (SaaS) ◾◾ TERADATA : http://www.teradata.com/products-and-services/cloud-overview/ Offers managed database services and also combines its database with other Cloud platforms, like AWS and Azure Index A Account(s), 435 management, 378, 380 retention, 437 sales, 437 ACID (atomic, consistent, isolated, and durable) database, 302 Active IoT devices, 107 ADC sensors, see Analog-to-digital (ADC) sensors ADM, see Architecture Development Method (ADM) Advertising, 377, 378, 380– 381 Agile, advantages of, 388– 390 analytic categories and, 114– 121 in analytics and storage, 192– 193 architectural change management and, 189– 190 and Big Data, 7– 8, 23– 24, 188, 380– 381 in business dimensions, 395– 396 in business functions, 21 business opportunities, 175– 176 as business value, 386– 388 CAMS balancing, 400– 403 and Cloud, 227, 231– 232 CMS and blogging, 306 collaborations and intelligence as functioning, 403– 409 collaborative partnerships in offerings, 85 conversational nature, 389 as customer-centric, rapidly changing business, 85 data and, 27 decision making with fine granularity, 24– 26 in DoE, 475– 477 embedding, 352 embedding Big Data in business processes, 40– 41 in enterprise architecture, 170– 171 enterprise’ s response, 25 envisioning, 390– 393 event logging and, 305– 306 expiring usage and, 306 external influencing factors business partners, 397 customer relationships, 396 government regulatory factors and sustainability, 397 sociocultural environment, 397– 398 facilitation of, 131 fine granularity, 113– 114, 393– 395 Hadoop and, 190– 192 holistic business, 86, 393– 395 holistic customer, 410– 411 implementation using, 173 in infrastructure, 21 in-memory NoSQL databases, 306– 307 internal influencing factors business compliance, 398 business innovation, 398 business structure, 398 people management, 399 product management, 399– 400 technology management, 399 job aids for, 147 key– value stores, 306– 307 Lean and, 27 learning, 351– 352 leveraging analytics, 119– 121 mobile apps and, 210– 212 NoSQL databases on, 231, 305– 306 organizations, types and sizes, 24 principles, 21, 401– 402 and Semantic Web, 260– 266 sensitive business, 393– 395 services development using, 239– 240 SMAC and, 202– 204 storage handling, 286 All-to-all (A2A) business model, 275 Amazon, 489 Amazon Web Services (AWS), 293, 299 Ambari, 480 Analog-to-digital (ADC) sensors, 167 Analytical skills, 349 Analytics, 200, 201 in banking, 432 and CAMS, 204 layer in EA, 182– 183 491 492 ◾ Index quality, 316 Analytics as a service (AaaS), 133, 235– 236 in banking, 434, 435 offering, 238 Apache Software Foundation, 479 Apex, 480 Application layer, in EA, 183 Application management, 342 Application support, 238 Architecture, and Big Data, 164– 165 Architecture centric Agile, 402 Architecture Development Method (ADM), 171 The Art of Agile Practice, 24 Aspirations, 437 Audit, 11, 381, 439 Auditors, 420 Availability management, 340 Avro, 480 AWS, see Amazon Web Services (AWS) B BA, see Business analysis (BA); Business architecture (BA) BABOK 3.0, 127 Backlog management, 148 Banking AaaS in, 434, 435 Agile, 433– 434 analytics in, 432 Big Data adoption immediate/tactical advantages for, 427 operations, 427– 428 strategic advantages, 428– 429 TESP subframework, 429 branded services, 438 Cloud in, 432 collaboration, 440 data governance issues, 440– 4 41 description of case study, 417– 418 enterprise architecture factors, 431 Hadoop-based technologies, 432– 433 list of opportunities, 418– 419 mapping of Big Data, 424– 426 mobility in, 432 operational services, 438– 439 privacy of data, 442 quality of shared services, 439– 4 40 retail banking services, 434– 436 security of data, 442 Semantic Web, 440 services, 436– 437 social media and, 429, 432 stakeholders, 419– 420 strategy, 433 SWOT analysis, 421– 424 veracity of data, 441– 4 42 BASE (basically available, soft state, and eventually consistent) database, 302– 303 BDFAB, see Big Data Framework for Agile Business (BDFAB) Behavioral analytics, 120 Big Data Agile and, 7– 8, 23– 24 analytics and visualization, 21, 100 application in developing Agile communities, 380– 381 architecture and, 164– 165 business factors impacting adoption of, 13– 14 business value from, 3– 4, 208– 210 capabilities, 24 and changing business functions business analysis, 136– 137 business process reengineering, 136– 137 change management, 136– 137 Lean approaches, 138 modeling requirements, 139– 140 nonfunctional/operational requirements, 143– 144 organizational information systems, 135– 136 UML, 142– 143 usability requirements, 144 use cases, 141– 142 changing business operations, 367– 368 Cloud characteristics and, 225 and collaborations, 274– 280 in community services, 378– 380 contemporary challenges, 31– 32 detailed process models, 32 lack of a holistic view, 33– 34 lack of standards and skills, 35 mobile banking process, 33 overwhelming and fast-changing technology, 34 understanding business parameters, 32– 33 volume, velocity, and variety (three Vs), 35 data-driven decisions, information, and knowledge, data points, 165 decision-making process, 5, 29– 30 differentiator, 6– 7 disparate elements and their synchronization through services, 186– 188 driven business strategies external growth, 28 internal optimization, 28– 30 risk management and compliance, 31 sustainability and environment, 31 ecosystem, 479– 484 embedding, 40– 41, 352 factors influencing formation of, 39 in finance, 10 finer granularity, 10– 11 5Vs of, 103– 104 GRC in, 334– 335 innovation, 21 Index ◾ 493 input/output, 100 intensified with, 165 IoT, 169 iterative adoption of, 159 Lean and, 27 learning, 351– 352 management and administration, 21, 100 challenges, 285– 287 transition at operational level, 369 manifesto, 62, 83– 84 mapping strategy to EA, 171– 173 maturity model, 71 operational advantages of, 36 organizational capacity and capabilities, 369– 373 organizational change with, 366– 367 outcomes and behaviors, 375– 376 performance metrics, 374– 375 quality and testing, 102 recruiting process, 375 regulations and compliance, 102 resourcing service model, 368– 369 role transition, 376 Semantic Web and, 259– 260 service capacity and capability building around, 242 change management and self-serve analytics, 242– 244 market development, 242 organic growth of, 242 positive experience to users, 241– 242 requirements, 238– 239 sharing and collaboration, 102 soft skills development, 376 software technical architecture from, 177 strategic approach, 266– 267 advantages of, 36– 37 analysis, 12– 16, 20 benefit, 25 development, 20 envisioning, 18– 23 foundations, 37 impetus and catalysts for, 37– 38 implementation, 20 layers, 22– 23 setting the scene for, 9– 12 short-and long-term decision making, 22 technical, analytical, and strategic decisions, 9– 10 strategy cube, 61, 80– 82 strengths of, 72– 73 tactical advantages of, 36 technical architecture, 176– 179 technologies, 21 tool selection, checklist built-in GRC, 486 context/metadata storage, 486 control movement of data, 485 costs, 486 ease of collaboration, 486 ease of creating services, 485 ease of use, 485 end-to-end development environment, 485 integration with existing enterprise data warehouse, 486 references, 486 support data quality, 485 usage, 205– 206 value-added strategies for, 35– 36 volumes of data, Big Data adoption, 10, 38– 42, 438 aligning, 160 analytical packages, 69 analytical skills, 349 business decision making, 349 business process modeling in change management, 130 changing face, 128– 129 continuous testing and showcasing, 130 elimination of redundancies, 131 embedded sustainability in operations, 131 facilitation of Agile, 131 GRC, quality, and audit support processes, 131 impact of Agile, 130– 131 importance, 126– 128 integration solutions, 130 learning, 131 range of processes in organization, 129 visibility, 130 CAMS in, 146– 149 DevOps and operationalizing solution, 149– 156 preiteration Agile practices in, 147– 149 requirements modeling, activities and tasks in, 149– 155 capability enhancement, 349– 352 change management costs, 69 costs, 69 data science, overlapping skills, 353 enterprise architecture in, 165– 168 hard and soft skills, 348– 349 opportunities with, 74– 75 process, 129 roadmap, 61, 80, 156– 159 skills gap, 348 technical skills, 348– 349 technologies, 68 threats from, 75– 76 weaknesses, 73– 74 Big Data analytics adopting and positioning on Cloud, 244– 245 and creative business processes, 145 self-service vs. managed service, 240– 241 steps in embedding analytics in processes, 145– 146 types of, 237– 238 494 ◾ Index Big Data Analytics (Sathi), 27 Big Data Framework for Agile Business (BDFAB) adoption framework, need for, 48– 50 artifacts/deliverables, 51, 60, 78 benefit of, 49 building blocks/modules, 51, 62 business decisions, 59, 63 business processes and granularity in decision making, 59, 63, 76– 77 data science (see Data science: analytics, context, and strategies) enterprise architecture, 59– 60, 63, 77 mini-iterations, 64– 66 quality, GRC, and people, 59, 63, 77– 78 business conditions, 60– 61, 78 drivers, 50 investment decision, 66– 67 and organizational environment, exploring, 67 strategy, 49 capacities and capabilities, 49 collaborations, 49, 54 communities formation, 86 compendium, 51, 61– 62, 80– 84 conditions, 51 customer experience, 85 deployment, 50 dynamicity, 58 in education, 468– 472 external skills, acceptance, 87 gap identification and change management, 49 governance, 58 and compliance, 87 and controls, 49– 50 holistic Agile business, 86 implementation of, 408 insights, 54 iterative and incremental process, 62 key elements of, 51– 54 KPIs for, 68 leanness, 58 multitiered analytics, 86 organizational readiness, assessing, 69– 70 overview of, 55– 57 people, focus on, 87 practices, 51 RACI matrix, 362– 366 reality testing with enterprise architecture, 85– 86 responsibilities, 49 risk and SWOT analysis, 70– 72 in transforming, 84 roadmap, 50 ROIs in, 68– 69 roles, 49, 51, 58– 59 service model, 50 stakeholders in, 140– 141 sustainability, 58 and carbon compliance, 87 users, iterative exploration of needs by, 84– 85 values, 51, 54– 58 Billing support system (BSS), 136 Binding layer, in EA, 182– 183 Blogging, 473 BPM, see Business process modeling (BPM) BPR, see Business process reengineering (BPR) Branding, 437 Bring your own device (BYOD), 203 Broadcasting business processes, 408 Business change, 240 class modeling, 153 compliance, 398 context, 173 decisions, 59 ecosystem, 394 innovation, 398 integration, 206, 368 investment, 438, 439 investment decision, 66– 67 leadership, 395 data science to, 17– 18 management, 368, 374 metrics, 395 needs, 367 partners, 397 planning, 370 policies, 394 processes, 129, 438, 439 changes to, 11 users and customers of, 140 risks, 394 strategy, 367 structure, 398 Business analysis (BA), 17, 136– 137 Business architecture (BA), 166 Business processes layer, in EA, 183 Business process management, and CAMS, 137– 138 Business process modeling (BPM), 133– 134 change management, 130 changing face, 128– 129 continuous testing and showcasing, 130 elimination of redundancies, 131 embedded sustainability in operations, 131 facilitation of Agile, 131 GRC, quality, and audit support processes, 131 impact of Agile, 130– 131 importance, 126– 128 integration solutions, 130 learning, 131 range of processes in organization, 129 visibility, 130 Business process reengineering (BPR), 136– 137 technology-enabled BPR, 138– 139 Index ◾ 495 Business– technology exploration, 64 Business Technology Office, 24 BYOD, see Bring your own device (BYOD) C CAMS, see Composite Agile Method and Strategy (CAMS) Capability, 242 analysis, 147 enhancement, 349– 352 management, 371 Capacity, 242 management, 340, 372 organizational capacity and capabilities, 369– 373 service capacity, 242 CAPEX, see Capital expenditures (CAPEX) Capital expenditures (CAPEX), 227 CAP theorem, see Consistency, availability, and partition tolerance (CAP) theorem Carbon footprints, 246– 248 Card retention, 436 Card sales, 436 Cash flow management, 437 Cassandra, 297, 487 CBPE, see Collaborative business process engineering (CBPE) Change management, 136– 137, 242– 244, 339 Chukwa, 480 CI, see Collaborative intelligence (CI) Cloud-based services platform, 377– 378 Cloud computing (Cloud), 59– 60, 133, 167, 201 adopting and positioning Big Data analytics, 244– 245 Agility and, 227, 231– 232 analytics as a service, 235– 236 architecting analytical services, 236– 237 architecture, 224– 225 in banking, 432 Big Data analytics challenges on, 228– 230 and CAMS, 204 characteristics, 225 collaborative analytics on, 228 connectivity, 225 as cost-effective mechanism for storage and analytics, 227 data analytics requirements on, 238– 239 data sharing on, 226 data storage and security, 225– 226 domain, 224– 225 and EA, 232– 233 infrastructure, 182 infrastructure as a service, 235 intersection and analytics with SoMo, 233– 234 leanness facilitated by, 227 MobileNerd, 236 platform as a service, 235, 489– 490 reducing carbon footprints, 246– 248 scalability/elasticity of, 227 self-service vs. managed service, 240– 241 services development, 239– 240 single-user view using, 227– 228 and SMAC stack, 224 and SMES, 248– 250 software as a service, 234 storage handling, 286 and sustainability, 245– 246 visualizations and, 228 Clusters, 275, 276 CMS, see content management system (CMS) Collaborations and Agility, 275– 276 banking, 440 business processes, 232, 407, 409 electronic collaboration, 404– 405 in electronic form, 406 environments and business value, 277– 278 horizontal clusters, 276 information, 406 intelligence, 407 knowledge, 407 mobile collaboration, 405 physical collaboration, 404 process, 406– 407 and self-serve analytics, 405– 406 understanding, 274– 275 vertical clusters, 276– 277 Collaborative business process engineering (CBPE), 275– 280 business integration with, 278– 280 in DoE, 477 Collaborative intelligence (CI), 405 Columnar databases, 295– 297 Communication layer, of EA, 180, 182 Community engagement, 377, 379 Community forums, 473 Community payments, 377, 379 Community services, 378– 380 Compliance, 11, 439 Composite Agile Method and Strategy (CAMS), 26– 27 analytics and, 204 balancing Agility, 400– 403 in Big Data adoption, 146– 149 DevOps and operationalizing solution, 149– 156 preiteration Agile practices in, 147– 149 requirements modeling, activities and tasks in, 149– 155 business process management and, 137– 138 cloud and, 204 importance of, 199– 201 mobile and, 203– 204 social media and, 202– 203 in solutions space, 402– 403 Computing, 4, 240 496 ◾ Index Conceptual exploration, 173 Configuration management, 338 Consistency, availability, and partition tolerance (CAP) theorem and NoSQL, 303– 304 sharding and replication in, 304 Consumption tracking, 341 Contemporary testing, 333 Content management system (CMS), 306 Context awareness, 106 Context modeling, 153 Continuous testing, 148, 330– 331 Corporate business strategy, 266 Couchbase, 487 Credit cards, 436 CRM system, see Customer relationship management (CRM) system Cross-disciplinary teams, 370 Crowd sourcing, 205, 232 Customer(s), 420 analytics, 119 focus, 372 intelligence, 99 management, 343 ownership, 438 relationships, 396 response, 13 Customer-centric Agile, 21, 401– 402 Customer-driven reengineering, 138 Customer relationship management (CRM) system, 135, 165, 212– 214 D Daily stand-up meeting, 148 Data, 3– 4, 177; see also Big Data and Agile, 27 analyst, 59 architect, 58 changes to, 11 and datum, 4– 5 history of, 4– 5 integration, 248 management, 100, 343 matching, 18 quality, 316, 330 and quality analysts, 420 security of, 34 sharing on Cloud, 226 Data analytics as core part of data science, 97– 98 designing, 98 strategic approach to, 99– 103 Database management, 342– 343 Data-centric approach, 98 Data-cum-analytics (DatAnalytics), 99 Data-driven decisions, Data point additional free space provisioning, 105, 106 backup, 105 and context, 104– 108 context-based data point, 110– 111 custom generation, 106 mirroring, 105 provisioning, 106 quality and reliability, 105 security, 105 speed and density, 105 Data science, 59, 438, 439 to business leadership, 17– 18 overlapping skills of, 353 Data science: analytics, context, and strategies, 59, 63, 76 analytic categories, 114– 121 context-based data point, 110– 111 data analytics as core part, 97– 98 data curiosity by business, 95– 97 data life cycle, 102 data point and context, 104– 108 for data transformation, 96 fine granularity and Agile, 113– 114 granularity, 111– 112 hex elementization, 108– 110 importance, 93– 95 leading and lagging indicators, 116– 117 leveraging analytics, 119– 121 machine learning, 108– 110 security and storage issues, 104 self-serve analytics, 99 strategic approach to data analytics, 99– 103 types and characteristics, 103– 104 Data scientist, 58, 420 SFIA, 357– 359 Data storage, 167 and business decisions, 284– 285 on Cloud, 225– 226 in EA, 182 Dating sites, 380 Datomic database, 487 Decentralized decision making, 130 Decision makers, Decision-making process, 5, 349 business processes and granularity in, 59 collaborative, 29 crowd sourcing, 30 data sources merging, 29 decentralized, 29, 130 dynamicity, 30 finer granularity in, 68 lack of balance in, 75 real time, 29 self-service, 29 Department of Education (DoE) agility in, 475– 477 Index ◾ 497 BDFAB iterations, creating, 468– 472 Big Data business case for, 465 quality, 477– 478 risks from incorporation, 474– 475 Big Data adoption, advantages and risks, 473– 475 finances and ROI in education, 466 government scenario, 463– 465 immediate/tactical advantages, 474 operational advantages, 474– 475 SMAC, 473 stakeholders of, 468 strategic advantages, 474 SWOT analysis, 466– 468 volume, velocity, and veracity, 472 Descriptive analytics, 118 Developer-centric Agile, 21, 402 Development and operations (DevOps), 149– 156, 338– 339 Diagnostic analytics, 118 Directory search, 378– 380 Directory services, 381 Document-centric databases, 293– 294 DoE, see Department of Education (DoE) Domain analysis, 151 Drill, 481 DynamoDB, 487 E EA, see Enterprise architecture (EA) EC2, see Elastic Compute Cloud (EC2) e-CRM, see Electronic customer relationship management (e-CRM) EI, see Environmental intelligence (EI) Elastic Compute Cloud (EC2), 299 Electronic collaboration, 404– 405 Electronic customer relationship management (e-CRM), 212 Electronic patient records (EPRs), 453– 457 End-to-end processes, 371 Enterprise architecture (EA), 5, 18, 19 Agility in, 170– 171 and analytics, 65 banking, 431, 438, 439 BDFAB, 48, 59– 60 in Big Data technology adoption, 165– 168 and business architecture, 166 Cloud and, 232– 233 mapping Big Data strategy to, 171– 173 robustness, 168 stack layers analytics and binding, 182– 183 business processes and applications, 183 communications (networks and infrastructure), 180, 182 data storage (SQL and NoSQL), 182 presentations and visualization, 184 360° hospital application, 184 Enterprise engineering (EE), 169 Enterprise IT strategy, 267– 268 Enterprise resource planning (ERP) system, 136, 165 Environmental intelligence (EI), 245– 246 Environment management, 343 EPRs, see Electronic patient records (EPRs) ERP system, see Enterprise resource planning (ERP) system Estimation, Agile practices, 148 ETL tools, see Extract, transform, and load (ETL) tools Executive and board remuneration, 14 Explicit knowledge, 38 Explorative analytics, 118 External business processes, 129 External customer and partner relationship strategy, 267 External service desk, 241 External stakeholders, 140 Extract, transform, and load (ETL) tools, 175, 179 Extreme programming (XP), 386 F Facebook, 473 Finances, in education, 466 Financial management, 341, 368, 374 Finer granularity and Agile, 113– 114 analytics, 98, 190 in business response, 131 context-based fine granularity, 114 of data and analytics, 111– 112 in decision making, 68 Flume, 481 Foreign exchange, 436 Foreign exchange retention, 437 Foreign exchange sales, 437 Functional testing, 328, 333 Fundraising, 377– 379 G Google Cloud Platform, 489 Governance, complexity and lack of, 35 Governance– R isk– Compliance (GRC), 60 application management, 342 audit, 341 availability management, 340 balancing act, 336– 337 in Big Data, 334– 335 capacity management, 340 change management, 339 characteristics, 337 configuration management, 338 customer management, 343 database management, 342– 343 498 ◾ Index data management, 343 environment management, 343 financial management, 341 implementation, 336 incident management, 338– 339 legal and compliance, 136 metrics and measurement, 343– 344 problem management, 339 release management, 339 request management, 342 risk, 341 security compliance, 341 service continuity management, 340 service desk, 338 service-level management, 340– 341 service support using ITIL, 337– 338 technology benefits, 336 Government regulatory factors, 397 GPS navigation system, 264 Graph databases, 294– 295 GRC, see Governance– R isk– Compliance (GRC) H Hadoop, 6, 14, 23, 48, 173 and Agility, 190– 192 analytical, storage, and infrastructure technologies enabled by, 178, 179 basics of, 173– 174 Big Data storage, 288 business opportunities, 175– 176 MapReduce, 179 SMAC stack integrated with, 207 Spark, 179– 180 storage handling, 285 Hadoop Distributed File System (HDFS), 166, 174 architecture, 102 NoSQL databases, 62 HBase, 102, 297, 481, 487 HCatalog, 482 HDFS, see Hadoop Distributed File System (HDFS) Health domain Big Data technology stack, 458 business processes of, 452– 453 capturing quality data, 459 description of case study, 445– 4 46 electronic patient records, 453– 457 people skills and capabilities, enhancing, 459, 462 quality, privacy, and security issues, 459 Semantic Web and analytics, 458– 459 SFIA skill, 460– 461 SMAC stack in, 457– 458 stakeholders in, 449 strategic value, 450 SWOT analysis of, 447– 4 48 volume and velocity, 449, 451 Health Insurance Portability and Accountability Act (HIPAA), 318 Hex elementization, 108– 110 Higher-level analytics, 113 HIPAA, see Health Insurance Portability and Accountability Act (HIPAA) Hive, 102, 482 HP Enterprise, 489 Hue, 482 Human resources (HR) management, 136, 268 I IaaS, see Infrastructure as a Service (IaaS) IBM, 489 ICT, see Information and communications technology (ICT) IEEE, see Institute of Electrical and Electronics Engineers (IEEE) Impala, 482 Incident management, 240, 338– 339 Incumbency, 14 Information, 8, 12, 22 Information and communications technology (ICT) challenges for, 368– 369, 374 changes to, 11 implementation, 48 operations, changing, 376 Information Technology Infrastructure Library (ITIL), 337– 338 Informative analytics, 117 Informative business processes, 408 Infrastructure as a Service (IaaS), 235 Infrastructure layer, of EA, 180, 182 Instance modeling, 154 Institute of Electrical and Electronics Engineers (IEEE), 164 Insurance, 436 Insurance retention, 437 Insurance sales, 437 Intelligence, 23 Internal business processes, 129 Internal service desk, 241 Internal stakeholders, 140 Internet-based exchange, 268 Internet of Everything (IoE), 169 Internet of Things (IoT), 8, 169 as basis for data points, 107 high-volume data, 12 storage handling, 286 and waves of high-velocity data, 169– 170 Investment cost, vs. opportunity, 13 Investor, 58 IoE, see Internet of Everything (IoE) IoT, see Internet of Things (IoT) Iteration planning, 147 Index ◾ 499 ITIL, see Information Technology Infrastructure Library (ITIL) J Jaql, 482 JavaScript Object Notation (JSON), 264 Java Virtual Machine (JVM), 180 JSON, see JavaScript Object Notation (JSON) JVM, see Java Virtual Machine (JVM) K Key performance indicators (KPIs), 50, 68 Key– value pairs (KVPs), 293 Knowledge, 8, 23 KPIs, see Key performance indicators (KPIs) L Lean approaches and Agile, 27 business and IT, 138 facilitated by Cloud, 227 large-scale processes, 138 Loan, 436 Loan retention, 436 Loans sales, 436 Lucene, 482 M Machine-generated data, 232, 286 Machine learning (ML), 108– 110, 178 Mahout, 483 Maintenance analytics, 120 Managed investment, 436 Managed investment retention, 437 Managed investment sales, 437 Managed service, 240– 241 Management capability, 371 Management capacity, 372 Management centric Agile, 402 Management quality, 316 MapReduce, 174, 175 MapReduce algorithm, 8, 102, 166 MapReduce-based programming languages, 62 Marketing analytics, 119 MarkLogic, 487 Massive parallel processing (MPP), 173 Mass personalization, 232 Master data management (MDM), 33– 34, 410 Matching, 381 m-CRM, see Mobile customer relationship management (m-CRM) MDM, see Master data management (MDM) Mentor, 58 Microsoft Azure, 489 ML, see Machine learning (ML) Mobile apps and CAMS, 203– 204 development and deployment, 210– 211 in DoE, 473 dynamic business processes, 213 dynamic customer group “ tribe” formation, 214 in health domain, 457– 458 personalization, 211– 212 real-time interaction with, 212– 213 Short Message Service, 213 spot-based analytics, 213 user preferences, 212 Mobile banking process, 33 Mobile collaboration, 405 Mobile customer relationship management (m-CRM), 212– 214 Mobile data, storage handling, 285– 286 MobileNerd, 236, 489 Mobility, 200, 201, 432 Model quality, 316 Model– View– Controller (MVC), 176 Monetized service, 341 MongoDB, 294, 487 Mortgages, 436 MPP, see Massive parallel processing (MPP) Multimedia data, in SAAs, 261 N Neo4J, 487 Net promoter score (NPS), 14, 118, 391 Network, 239 Network layer, of EA, 180, 182 Nonfunctional requirements, 143– 144 Nonfunctional testing, 329 NoSQL databases, see Not Only Structured Query Language (NoSQL) databases Not Only Structured Query Language (NoSQL) databases, 5, 6, 98, 167, 173 ACID, 302 on Agile, 231, 305– 306 BASE, 302– 303 and Big Data, 290– 291 business decisions, 284– 285 and business value, schemalessness of, 291– 293 CAP theorem and, 303– 304 clustering, 301– 302 columnar databases, 295– 297 with commercial options, 487 comparison factors, 297– 300 data storage, 102, 284– 285, 287– 288 distribution, 301– 302 document-centric databases, 293– 294 in EA, 182 graph databases, 294– 295 500 ◾ Index in-memory storage, 306– 307 KVPs, 293 MongoDB, 294 semi-and unstructured data, handling, 288– 290 sharding, 301– 302 using in practice, 300– 301 NPS, see Net promoter score (NPS) O OAT, see Operational acceptance testing (OAT) Object-oriented (OO) databases, Observations, 22 ODIs, see Open data interfaces (ODIs) Onboarding, 438 Online shopping, 145 Ontologies and rules, 272 and taxonomies, 268– 271 Oozie, 483 Open data interfaces (ODIs), 24 Open-source community, 479 Operating systems, 240 Operational acceptance testing (OAT), 331 Operational analysis, 155 Operational expenditures (OPEX), 227 Operational requirements, 143– 144 Operational risk, 13 Operational support, 240 Operations and support system (OSS), 136 Operative business processes, 409 OPEX, see Operational expenditures (OPEX) Optimal granularity level (OGL), 111, 112 Oracle, 487, 489 Organic networks, 242 Organizational information systems, 135– 136 P PaaS, see Platform as a Service (PaaS) Partners, 59, 420 Pattern matching, 18 Payment history, 437 People management, 11, 66, 399 Physical collaboration, 404 Pig architecture, 102, 483 Platform as a Service (PaaS), 235 PLM, see Product life cycle management (PLM) Predictive analytics, 118 Prescriptive analytics, 118 Presentation, 179 Presentation layer, in EA, 184 Presentation technologies, 167 Prioritization, 148 Private Cloud, 226 Problem management, 339 Process models, 59, 173 Process quality, 316, 330 Product analytics, 119 Product life cycle management (PLM), 400 Product management, 399– 400 Product specialists, 438 Product standards, 439– 4 40 Project management, 240, 373 Proven process, Public Cloud, 226 Python, 102 Q Quality of Big Data, 65, 438, 439 adoption, 317– 318 aesthetics and ease in use, 326 analyst, 59 analytics, 315– 316 business processes, 317 cleansing and staging, 322– 323 considerations, 314– 315 contemporary testing, 333 continuous testing, 330– 331 of data entry, 321– 322 data retirement, 323 detection vs. prevention, 314– 315 domain, 315 environment, 316 functional testing, 333 functional vs. nonfunctional quality, 328– 329 inherent and applied data, 318– 319 issues, 314 management, 317 metadata, 329 model and architecture, 316– 317 practices, 326– 328 semantic meaning, 326 of shared services, 439– 4 40 sifting value from noise, 329– 330 strategic considerations, 319– 320 syntactical correctness, 325– 326 and testing, 323– 324 transition phases, 320– 323 variety testing, 332 velocity testing, 332– 333 verification and validation, 324– 325 visualizations, 334 volume testing, 332 R RACI matrix, 362– 366, 419– 420 Radio frequency identification (RFID), 190 Reactive analytics, 118 Real-time decision making, 237 Redis, 487 Regulators, 420 Index ◾ 501 Regulatory, 14, 381 Release management, 339 Request management, 240, 342 Requirement analysis, 154 envisioning, 151 Resource Description Framework (RDF), 272– 274 Retrospective, 149 Return on investment (ROI), 68– 69, 466 Rewards, 436 Rewards sales, 437 RFID, see Radio frequency identification (RFID) Riak, 487 Risk management, 11 ROI, see Return on investment (ROI) R project (Apache projects), 102, 483 S SAAs, see Semantically aware applications (SAAs) SaaS, see Software as a Service (SaaS) Sales analytics, 120 SAP, 489 Sarbanes– Oxley Act (SOX), 318 SAS, 490 SCM, see Supply chain management (SCM) Scribe, 483 Scrum, 386 Searching, 381 Security architecture, 184– 186 changes to, 11 on Cloud, 225– 226 compliance audit risk, 368 Self-managed investment, 436 Self-managed investment retention, 437 Self-managed investment sales, 437 Self-serve analytics (SSA), 34, 58, 99, 242– 244 architecting, 237 collaborations and, 405– 406 quality of, 324 Self-service, 240– 241 Semantically aware applications (SAAs) business value of, 271– 272 development phase, 262, 263 multimedia data in, 261 resource description framework, 272– 274 Semantic Web, 59– 60, 63, 167 Agility and, 260– 266 banking, 440 and Big Data, 259– 260 communities, 269 data types, 265 information and knowledge exchange, 268, 269 key elements of, 259 knowledge generation in, 268– 271 ontologies and taxonomies, 268– 271 and organizational strategies, 266– 268 Semantic Web technologies (SWTs), 256– 257, 272– 274 Semistructured data, 232, 274 Senior management, 420 Sentiment analytics, 120 Servers, 239– 240 Service(s), 177, 381 continuity management, 340 costing, 341 delivery, 367 desk, 240 development, 238 improvement, 238 management, 368, 374 model, 372 support, 238, 368 Service-oriented architecture (SOA), 34, 262 SFIA, see Skills Framework for Information Age (SFIA) Short Message Service (SMS), 213 Showcasing, 149 Simple Protocol and RDF Query Language (SPARQL), 273– 274 Single-user view, using Cloud, 227– 228 Skills Framework for Information Age (SFIA) business skills, 358 data scientist, 357– 359 developing team, 355– 356 enhancing organizational capabilities, 359, 362 governance, quality, and testing skills, 361 health domain, 460– 461 mapping to Big Data skills, 353– 355 organizational capabilities, 355– 356 technical skills, 360 training and upskilling resources, 357 SMAC stack, see Social, mobile, analytics, and Cloud (SMAC) stack Small and medium enterprises (SMEs), 24, 133 Cloud and, 248– 250 Smalltalk, 176 SMEs, see Small and medium enterprises (SMEs); Subject matter experts (SMEs) SMS, see Short Message Service (SMS) SOA, see Service-oriented architecture (SOA) SOAs, see Solutions-oriented architectures (SOAs) Social media, 200, 201, 205 and banking, 429, 432 business value from, 208– 209 and CAMS, 202– 203 and customer sentiments, 209 in DoE, 473 in health domain, 457 in practice, 210 Social media and mobile (SoMo), 59– 60, 182 Cloud intersection and analytics with, 233– 234 conceptual mapping, 197– 199 harnessing variety of data from, 209– 210 presentation/visualizations, 214– 215 502 ◾ Index storage handling, 286 sustainability and environment, 206 Social, mobile, analytics, and Cloud (SMAC) stack, 63, 77, 195 and Agile, 202– 204 banking and, 429– 432 and business integration, 206 and business size and type, 206– 207 business value from, 208– 210 conceptual mapping, 197– 199 consumers, 205 core elements of, 200 data from multiple sources and in multiple formats, 205 Department of Education, 473 elements, 196– 197 in health domain, 457– 458 and industry verticals, 210, 211 interconnected nature of, 199– 201 knowledge sharing across organization, 205– 206 providers, 205 risks and business concerns, 208 scalability and agility through Cloud solutions, 206 technologies and domains, 196– 197 user contact, 217– 219 user experience, 215– 217 value adders, 205 Sociocultural environment, 397– 398 Software as a Service (SaaS), 234 Soir, 484 Solution developer, 59 Solutions-oriented architectures (SOAs), 179 SoMo, see Social media and mobile (SoMo) SOX, see Sarbanes– Oxley Act (SOX) Spark, 102, 179– 180, 484 SPARQL, see Simple Protocol and RDF Query Language (SPARQL) Spot-based analytics, 213 SQL, see Structured Query Language (SQL) Sqoop, 179, 484 SSA, see Self-serve analytics (SSA) Stakeholders of banking, 419– 420 in BDFAB, 140– 141 of DoE, 468 in health domain, 449 Storm, 484 Storyboarding, 152 Strategic risk, 13 Strategic risk management, 370 Strength, weakness, opportunities, and threat (SWOT) analysis, 50 of banking, 421– 424 in education, 466– 468 of health domain, 447– 4 48 risk and, 70– 72 Structured data, 232 Structured Query Language (SQL), 5, 182 Subject matter experts (SMEs), 420 Superannuation, 436 Superannuation retention, 437 Superannuation sales, 437 Supply chain management (SCM), 135– 136, 268 Sustainability, 136, 397 SWOT analysis, see Strength, weakness, opportunities, and threat (SWOT) analysis SWTs, see Semantic Web technologies (SWTs) T Tacit knowledge, 38 Team formation, 147 Technical, economic, social, and process (TESP) dimensions, 63, 77 and Big Data dimensions, 132– 133 economic considerations, 133 process dimension, 133– 134 social, 134– 135 subframework and business processes, 132 technologies, 133 subframework, 429, 430 Technical skills, 348– 349 Technology– a nalytics Hadoop, 64 Technology-enabled BPR, 138– 139 Technology management, 100, 102, 399 Technology quality, 330 TERADATA, 490 TESP dimensions, see Technical, economic, social, and process (TESP) dimensions Testing, 240 Texting, 377 The Open Group Architecture Framework (TOGAF), 171 TOGAF, see The Open Group Architecture Framework (TOGAF) Transactive business processes, 408– 409 Twitter, 473 U UAT, see User acceptance testing (UAT) UML, see Unified Modeling Language (UML) Unified Modeling Language (UML), 5, 76, 142– 143, 169 Unstructured data, 232, 274, 286 Upskilling strategy, 268 Usability requirements, 144 Use case diagrams, 152 for functional testing, 333 in modeling requirements, 139– 144 User acceptance testing (UAT), 331 User-based modeling, 127 Index ◾ 503 User/customer, 59, 420 experience and SMAC stack, 215– 217 features, 148 persona, 148 storage handling, 286 User experience analysis (UXA), 215– 219 User interface (UI), 144 UXA framework (UXAF), 216– 219 W V Y Variety testing, 332 Velocity testing, 332– 333 Vendor management, 240, 368, 372, 374 Virtualization, reducing carbon footprints, 246– 248 Visible charting, 148 Visualization, and Cloud, 228 Visualization layer, in EA, 184 Volume testing, 332 YARN, 484 Workload management, 368, 374 X XML, 264 Z Zachman framework, 171, 450 Zika virus, 183 ZooKeeper, 102, 484 ... I INTRODUCTION TO BIG DATA STRATEGIES AND OUTLINE OF BIG DATA FRAMEWORK FOR AGILE BUSINESS (BDFAB) Introduction to BIG Data and Agile Business .3 Chapter Objectives Big Data and Business. . .Big Data Strategies for Agile Business Framework, Practices, and Transformation Roadmap Big Data Strategies for Agile Business Framework, Practices, and Transformation Roadmap ... Information 44 Big Data Framework for Agile Business (BDFAB) 47 Chapter Objectives 47 Big Data Framework for Agile Business 48 Need for a Framework for