SEO Warrior SEO Warrior John I Jerkovic Beijing • Cambridge • Farnham • Kưln • Sebastopol • Taipei • Tokyo SEO Warrior by John I Jerkovic Copyright © 2010 John I Jerkovic All rights reserved Printed in the United States of America Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472 O’Reilly books may be purchased for educational, business, or sales promotional use Online editions are also available for most titles (http://my.safaribooksonline.com) For more information, contact our corporate/institutional sales department: 800-998-9938 or corporate@oreilly.com Editor: Mike Loukides Production Editor: Sarah Schneider Copyeditor: Audrey Doyle Proofreader: Sarah Schneider Indexer: Lucie Haskins Cover Designer: Karen Montgomery Interior Designer: David Futato Illustrator: Robert Romano Printing History: November 2009: First Edition O’Reilly and the O’Reilly logo are registered trademarks of O’Reilly Media, Inc SEO Warrior, the image of an Eleonora’s falcon, and related trade dress are trademarks of O’Reilly Media, Inc Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks Where those designations appear in this book, and O’Reilly Media, Inc was aware of a trademark claim, the designations have been printed in caps or initial caps While every precaution has been taken in the preparation of this book, the publisher and author assume no responsibility for errors or omissions, or for damages resulting from the use of the information contained herein TM This book uses RepKover, a durable and flexible lay-flat binding ISBN: 978-0-596-15707-4 [M] 1257537191 Table of Contents Preface xv The Big Picture SEO Benefits SERP Real Estate The Trust Factor Lower Cost of Ownership SEO Challenges Competition No Guarantees Ranking Fluctuations Time Factors Organizational Structure The SEO Process The Research Phase The Planning and Strategy Phase The Implementation Phase The Monitoring Phase The Assessment Phase The Maintenance Phase SEO Alternatives Paid Ads (or Links) Traditional Marketing Summary 2 6 8 10 12 14 15 16 17 17 17 18 18 Search Engine Primer 19 Search Engines That Matter Today: Google Yahoo! Bing Types of Search Engines and Web Directories First-Tier Search Engines 20 21 22 23 23 v Second-Tier Search Engines Regional Search Engines Topical (Vertical) Search Engines Web Spider–Based Search Engines Hybrid Search Engines Meta Search Engines Web Directories Search Engine Anatomy Spiders, Robots, Bots, and Crawlers The Search (or Query) Interface Search Engine Indexing Search Engine Rankings Summary 23 24 24 24 25 25 26 27 27 30 31 32 33 Website Essentials 35 Domain Name Options Domain Name Namespaces Buying Domain Names Hosting Options Choosing Platforms Hosting Types Custom Site Design or Third-Party Software Employing Custom Development Utilizing Free or Paid Software Website Usability General Considerations Linking Considerations Know Your Demographics Ensure Web Browser Compatibility Create Simple Page Layouts Use Intelligent Page Formatting Create Smart HTML Forms Optimize Your Site for Speed Test Your Interface Design for Usability Website Accessibility Summary 36 36 39 44 45 46 49 49 54 56 57 57 57 58 58 59 59 59 60 60 60 Internal Ranking Factors 63 Analyzing SERPs On-Page Ranking Factors Keywords in the Tag Keywords in the Page URL Keywords in the Page Copy vi | Table of Contents 63 64 64 66 67 Keywords in the Description Tag Keywords in the Heading Tags Keyword Proximity Keyword Prominence Keywords in the Link Anchor Text Quality Outbound Links Web Page Age Web Page Size On-Site Ranking Factors Domain Name Keywords Size or Quantity of Content Linking Considerations Freshness of Pages Putting It All Together Running the Script Final HTML Report Summary 67 69 70 70 71 72 73 73 74 75 76 77 80 80 80 81 83 External Ranking Factors 85 External Links Know Your Referrers Quantity and Quality of External Links Broken Outbound Links Handling Broken Links User Behavior Patterns Analyzing the Search Engine Query Interface Google Analytics Google Toolbar User Behavior Lessons Website Performance and Website Age Website Performance Website Age Summary 85 86 88 92 92 93 94 95 95 97 97 97 99 100 Web Stats Monitoring 101 Web Server Logs and Formats NCSA Common Format NCSA Combined Format NCSA Combined Format in Apache Converting IIS W3C to NCSA Combined Spotting a Web Spider in Web Logs Popular Web Stats Tools Using WebLog Expert 101 102 103 104 104 104 105 106 Table of Contents | vii Number of Visitors Unique Versus Total Visitors Number of Hits Number of Page Views Referrers Search Engine (Referral) Hits Searches/Keywords Web Spider Hits Using AWStats Using Webalizer Tying Searches to Individual Web Pages Web Spider Patterns User Patterns Filtering Specific Data Types of Web Page Elements Summary 106 106 107 107 108 109 110 110 110 112 116 117 118 120 120 121 Google Webmaster Tools and Google Analytics 123 Google Webmaster Tools Webmaster Tools Setup Dashboard The “Site configuration” Section The “Your site on the web” Section The Diagnostics Section Google Analytics Installation and Setup Navigating Google Analytics Dashboard Traffic Sources Content Goals Google Analytics Shortcomings Summary 123 123 124 124 129 131 133 134 135 135 140 140 141 144 145 Search Engine Traps 147 JavaScript Traps JavaScript-Generated Content JavaScript Dynamic Links and Menus Ajax Dynamic Widget Traps Using Flash Using Java Applets Using ActiveX Controls viii | Table of Contents 147 147 150 152 154 154 155 156 creating a Google account, 336–337 keyword match types, 345–348 navigating through, 337 overview, 329, 330 setting up, 336–338 testing, 350–354 usage tips, 354–355 Google AdWords Keyword Tool additional information, 72, 229 finding competitor keywords, 269 overview, 67, 230 search volume considerations, 214 Google Analytics assessing current situation, 242 Benchmarking page, 138 Browser capabilities option, 139 Content section, 140 Dashboard page, 135–139 defining funnels, 143 defining goals, 141–143 establishing keyword baseline, 223 FeedBurner tool and, 257 installation and setup, 134 Languages page, 139 Map Overlay page, 139 navigating, 135 Network properties option, 139 New vs returning page, 139 overview, 123, 133 shortcomings, 144–145 tracking user behavior patterns, 95 Traffic Sources section, 140 viewing goal stats, 144 Visitor loyalty menu item, 139 Visitor recency page, 139 Visitor trending option, 139 Visitors Overview page, 137 Google Base, 302 Google bombing, 71 Google Content Network, 341, 355 Google filetype:txt command, 183 Google Images, 301 Google inanchor: command, 238, 359 Google Insights for Search tool, 232 Google intitle: command, 238, 359 Google inurl: command, 238, 359 Google link: command, 86, 271 Google Local Business Center, 260, 264 Google News, 301 458 | Index Google PageRank algorithm backlink checkers and, 271 defined, 32 distributed internal link popularity, 79 gov/.edu domains, 90 keyword-based competitor research, 265 splitting effect on links, 92 URL canonicalization and, 80 web directories and, 91, 258 Google Product Search platform, 302 Google Reader, 253, 310 Google related: command, 259, 265 Google search engine additional information, 21 background, 20 canonical link element, 294 DMCA and, 300 dominance of, 19 Flash support, 155 guidelines for, 362 keyword tools and resources, 229–233 long-term content, 292 meta search engines and, 26 paid links and, 17 requesting site reevaluation, 373 SOAP API toolkit, 282 social media sites and, 324 specific directives, 188 typical SERP layout, 2–3 Universal Search model, 25 web directories and, 258 YouTube videos, 311 Google Sets tool, 229, 231 Google site: command Bit.ly URL shortening service, 308 obtaining relative index size, 277 obtaining site size, 77 viewing domains, 37 Google Sitelinks feature, 126 Google Sitemap Checker, 205 Google Sitemap Generator, 204 Google Sitemap Protocol (see XML Sitemaps) Google Toolbar, 95 Google Trends expected buzz, 291 overview, 229, 231, 288 unexpected buzz, 289 Google Webmaster Tools assessing current situation, 242 Change of address subsection, 126 Crawl errors page, 131 Crawl stats page, 131 Crawler access option, 126 Dashboard page, 124, 205 Diagnostics section, 131–133 duplicate content URLs, 296 establishing keyword baseline, 223 Google Sitelinks feature, 126 HTML suggestions page, 132 Internal links page, 130 Keywords page, 130 Links to your site option, 129 overview, 123 Settings option, 127 setup, 123 Site configuration section, 124–128 Sitemap submissions, 205–207 Sitemaps option, 124, 206 Subscriber stats page, 130 Top search queries option, 129 validation tools, 177 viewing backlinks, 86 Your site on the web section, 129–130 Googlebot web spider description, 178 file types supported, 29 showing activity breakdown, 131 signature example, 104 typical request, 28 Googlebot-Image spider, 104, 178 Googlebot-Mobile spider, 178 gov domains, 90 greater than sign (>), 198 grep system command, 196 Gross, Bill, 330 gTLDs (generic top-level domains) governance model, 36 overview, 36–38 sponsored, 37 H tag canonical link element, 294 HTML meta directives, 187–189 heading tags, keywords in, 69 hidden links, 370 hidden text, 363, 364–367 Hilltop Algorithm, 89 HitTail tool, 270 Hitwise Intelligence study, Hitwise tool, 272 hosting providers buying services, 43, 44 choosing platforms, 45–46 collocation hosting, 48 comanaged hosting, 48 dedicated server hosting, 48 free hosting, 47 hosting types, 46–49 internal hosting, 49 managed hosting, 48 shared hosting, 47 web page caching, 166 web server compression, 164 hot link prevention, 300 HotScripts website, 261 htaccess file document security, 173 error pages, 166 handling website maintenance, 175 hot link prevention, 300 HTTP header directives, 189 redirect mechanism, 256 redirecting domain misspellings, 297 trailing slash, 296 URL variables and, 167 WWW prefix, 297 HTML forms, 59 HTML meta directives, 187–189 HTML reports, 81, 82 HTML Sitemaps, 195–197 HTML tags (see specific tags) HTML traps complex HTML, 163 external DIV tags, 160 extremely large pages, 162 formatting problems, 163 frames, 156–158 graphical text, 161–162 iframes, 158–160 HTML validation, 163 htpasswd file, 174 HTTP header directives, 189 HTTrack Website Copier, 277 Hulu website, 248 hybrid search engines, 25 Index | 459 I ICANN (Internet Corporation for Assigned Names and Numbers), 36 iframes (inline frames), 158–160 iGooMap tool, 204 IIS case insensitivity, 183 NCSA Combined format and, 104 URL rewriting, 54 implementation phase (SEO process), 14, 15 inbound links (see external links) index directive, 187 indexing crawling comparison, 172, 249 deep-linked content, 299 defined, 31, 172 fine-grained, 298 frames and, 157 hidden DIV tags and, 149 prohibiting, 172–176 social media sites, 324 Twitter support, 307 infinite loops, 295 inlinksAnalysis.pl script, 87 inner page invisibility, 78 internal hosting, 49 internal links, 77, 78 internal optimization, 15 internal ranking factors analyzing SERPs, 63 defined, 63 on-page ranking factors, 64–74 on-site ranking factors, 74–80 putting it all together, 80–82 Internet Archive site background, 273 blocking, 183, 185 Internet Corporation for Assigned Names and Numbers (ICANN), 36 iProspect survey, iTunes platform, 255, 313, 314 J Jaiku website, 324 Java applets, 155, 246 JavaScript language Ajax and, 152–154 dynamic links and menus, 150–152 460 | Index dynamic text within, 147–149 Google Analytics support, 144 sneaky redirects, 363 Joomla! open source software, 55, 294 K Kanoodle platform, 330 Kartoo.com search engine, 26 KeyCompete tool, 270 Keyword Dashboard tool, 376 Keyword Discovery tool, 237 Keyword Elite tool, 237 keyword forecast tool, 235 keyword group detection tool, 234 keyword modifiers, 219–221 keyword mutation detection tool, 234 keyword prominence, 70 keyword proximity, 70 keyword rankings (see rankings) keyword research additional tools, 236–237 buying cycles, 288 buzz information, 288 compiling targeted draft list, 224–237 establishing current baseline, 223 estimating search volume, 239 evaluating keywords, 237–240 finalizing keyword list, 240 finding additional keywords, 268–271 future events, 288 implementing strategy, 240 keyword modifiers, 219–221, 225 keyword stemming, 217–218, 225 Latent Semantic Indexing, 221 Microsoft Office support, 226 predictive SEO, 287–289 process overview, 222 researching competitors, 264–268 SEO research phase, 12 strategy for, 214–217 keyword stemming, 218, 225 keyword stuffing, 363, 364 keywords AdWords match types, 345–348 AdWords tips, 355 average page densities, 67 brainstorming, 224 broad, 215 capitalization considerations, 350 choosing better, 359 domain name, 75–76 dynamic insertion, 349 evaluating, 237–240 exact matching, 76 finalizing list, 240 finding related, 225–226 in tag, 67–69 in heading tags, 69 in link anchor text, 71–72 in outbound links, 73 in page copy, 67 in page URL, 66 language and, 217–222 localized terms, 224 long tail, 215 narrow, 215–217 partial matching, 76 SEO tuning, 359 in tag, 64–66 web stat monitoring and, 110 KeywordSpy tool, 237, 270 Krug, Steve, 56 KwMap tool, 236 L language campaign options, 341 keywords and, 217–222 lastmod attribute tag (XML Sitemaps), 197, 199, 203 Latent Semantic Analysis (LSA), 222 Latent Semantic Indexing (LSI), 221 less than sign (