developing microsoft media foundation applications

www.it-ebooks.info www.it-ebooks.info Developing Microsoft ® Media Foundation Applications Anton Polinger www.it-ebooks.info Published with the authorization of Microsoft Corporation by: O’Reilly Media, Inc. 1005 Gravenstein Highway North Sebastopol, California 95472 Copyright © 2011 by Anton Polinger All rights reserved. No part of the contents of this book may be reproduced or transmitted in any form or by any means without the written permission of the publisher. ISBN: 978-0-7356-5659-8 1 2 3 4 5 6 7 8 9 LSI 6 5 4 3 2 1 Printed and bound in the United States of America. Microsoft Press books are available through booksellers and distributors worldwide. If you need support related to this book, email Microsoft Press Book Support at mspinput@microsoft.com. Please tell us what you think of this book at http://www.microsoft.com/learning/booksurvey. Microsoft and the trademarks listed at http://www.microsoft.com/about/legal/en/us/IntellectualProperty/ Trademarks/EN-US.aspx are trademarks of the Microsoft group of companies. All other marks are property of their respective owners. The example companies, organizations, products, domain names, email addresses, logos, people, places, and events depicted herein are ctitious. No association with any real company, organization, product, domain name, email address, logo, person, place, or event is intended or should be inferred. This book expresses the author’s views and opinions. The information contained in this book is provided without any express, statutory, or implied warranties. Neither the authors, O’Reilly Media, Inc., Microsoft Corporation, nor its resellers, or distributors will be held liable for any damages caused or alleged to be caused either directly or indirectly by this book. Acquisitions and Developmental Editor: Russell Jones Production Editor: Teresa Elsey Editorial Production: Online Training Solutions, Inc. Technical Reviewers: Anders Klemets and Matthieu Maitre Indexer: Lucie Haskins Cover Design: Twist Creative • Seattle Cover Composition: Karen Montgomery www.it-ebooks.info This book is dedicated to my parents for putting up with me. — Anton Polinger www.it-ebooks.info www.it-ebooks.info Contents at a Glance Introduction xiii CHAPTER 1 Core Media Foundation Concepts 1 CHAPTER 2 TopoEdit 11 CHAPTER 3 Media Playback 23 CHAPTER 4 Transcoding 61 CHAPTER 5 Media Foundation Transforms 97 CHAPTER 6 Media Foundation Sources 139 CHAPTER 7 Media Foundation Sinks 205 CHAPTER 8 Custom Media Sessions 247 CHAPTER 9 Advanced Media Foundation Topics 287 APPENDIX A Debugging Media Foundation Code 323 APPENDIX B COM Concepts 331 APPENDIX C Active Template Library Objects 339 Index 345 About the Author 361 www.it-ebooks.info www.it-ebooks.info vii Contents Introduction xiii Chapter 1 Core Media Foundation Concepts 1 Media Foundation Audio/Video Pipelines 2 Media Foundation Components 5 Data Flow Through a Media Foundation Pipeline 7 Media Foundation Topologies 9 Conclusion 10 Chapter 2 TopoEdit 11 Manual Topology Construction in TopoEdit 16 Capturing Data from External Sources 20 Conclusion 22 Chapter 3 Media Playback 23 Basic File Rendering with Media Sessions 25 Creating the Player 27 Initializing the Media Session 28 Media Session Asynchronous Events 31 Event Processing and Player Behavior 34 Building the Media Pipeline 43 Creating the Media Foundation Source 44 Building the Partial Topology 48 Resolving the Partial Topology 55 Conclusion 57 Class Listings 57 What do you think of this book? We want to hear from you! Microsoft is interested in hearing your feedback so we can continually improve our books and learning resources for you. To participate in a brief online survey, please visit: microsoft.com/learning/booksurvey www.it-ebooks.info viii Contents Chapter 4 Transcoding 61 The Transcode API. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .62 Creating a Transcode Prole 64 The Transcoding Session 74 Transcoding with the Source Reader 78 Creating a Source Reader and a Sink Writer 80 Mapping Sink Writer Streams 81 Intermediate Format Negotiation 84 The Target Transcode Media Type 88 The Source-Reader-to-Sink-Writer Loop 92 Conclusion 94 Class Listings 94 Chapter 5 Media Foundation Transforms 97 MFT Architecture Overview 98 Writing a Simple MFT 101 Stream Conguration Functions 101 Media Type Selection Functions 107 MFT Data Processing 113 Status Query and Event Functions 119 MFT Registration 121 Injecting Images into Video Frames 122 Uncompressed Video Formats 123 RGB to YUV Image Conversion 125 Frame Format Detection 128 UYVY Image Injection 130 NV12 Image Injection 132 Conclusion 133 Class Listings 134 www.it-ebooks.info [...]... application Chapter 5, Media Foundation Transforms,” Chapter 6, Media Foundation Sources,” and Chapter 7, Media Foundation Sinks,” discuss and show the design of core Media Foundation components used in media processing pipelines And finally, Chapter 8, “Custom Media Sessions,” and Chapter 9, “Advanced Media Foundation Topics,” describe more advanced concepts behind the MF platform and applications xiv  Introduction... icrosoft Media Foundation (MF) is Microsoft s new media platform in Windows, introduced in Windows Vista MF is intended as the primary media application development platform, superseding and replacing Microsoft DirectShow, Microsoft DirectX Media Objects, Microsoft Video for Windows, and all other previous media technologies MF gives you the ability to create advanced video and audio processing applications. .. If you want to develop Windows media applications, you will need to use the Media Foundation platform to access various components and hardware acceleration capabilities provided with Windows Developing Microsoft Media Foundation Applications provides an organized walkthrough of the MF system, giving the reader an overview of the core ideas necessary for designing MF applications This book will provide... procedure will be covered in more detail in Chapter 3, Media Playback.” 6  Developing Microsoft Media Foundation Applications www.it-ebooks.info Data Flow Through a Media Foundation Pipeline As mentioned earlier, data is passed between individual components in a topology in chunks or packets, usually called media samples Each media sample is an object with a data buffer, with a small segment of the... all sorts of effects and deal with different types of media 10  Developing Microsoft Media Foundation Applications www.it-ebooks.info C hapter 2 TopoEdit Manual Topology Construction in TopoEdit 16 Capturing Data from External Sources 20 O ne of the most important tools in the arsenal of a Microsoft Media Foundation (MF) developer is the manual topology construction... Foundation Components 5 Media Foundation Topologies 9 M icrosoft Media Foundation (MF) applications are programs that load and use various MF components and modules to process various media data streams Some MF applications are designed to simply play back video or audio files Others convert the media streams between different formats, store them... streams between different formats, store them in different files, and even send and receive media data over the Internet In this chapter, you will learn the basic terms and concepts used when discussing and considering Media Foundation applications Media Foundation applications break up the tasks necessary to process media data streams into multiple simple steps Each step is performed by a separate MF component... components that any Media Foundation program can use In this book, you will see how to load existing MF components, create your own MF components, arrange them into chains, and use them to process media streams By combining these MF modules, you will learn how to write applications that can play back different types of media files, and perform complex operations on media data Media Foundation Audio/Video... how to debug asynchronous Media Foundation applications and gives a brief overview of the MFTrace debugging tool Appendix B provides a quick refresher for basic COM concepts Finally, Appendix C demonstrates several common ATL objects used in every sample in the book Finding Your Best Starting Point in This Book The various chapters of Developing Microsoft Media Foundation Applications cover several... http://www .microsoft. com/learning/booksurvey The survey is short, and we read every one of your comments and ideas Thanks in advance for your input! Stay in Touch Let’s keep the conversation going! We’re on Twitter: http://twitter.com/MicrosoftPress xxii  Introduction www.it-ebooks.info C hapter 1 Core Media Foundation Concepts Media Foundation Audio/Video Pipelines 2 Media Foundation . xiii Chapter 1 Core Media Foundation Concepts 1 Media Foundation Audio/Video Pipelines 2 Media Foundation Components 5 Data Flow Through a Media Foundation Pipeline 7 Media Foundation Topologies. 5, Media Foundation Transforms,” Chapter 6, Media Foundation Sources,” and Chapter 7, Media Foundation Sinks,” discuss and show the design of core Media Foun- dation components used in media. Foundation Sources 139 CHAPTER 7 Media Foundation Sinks 205 CHAPTER 8 Custom Media Sessions 247 CHAPTER 9 Advanced Media Foundation Topics 287 APPENDIX A Debugging Media Foundation Code 323 APPENDIX