developers.google.com/search/docs/crawling-indexing/robots/create-robots-txt

Preview meta tags from the developers.google.com website.

Linked Hostnames

15

Thumbnail

Search Engine Appearance

Google

https://developers.google.com/search/docs/crawling-indexing/robots/create-robots-txt

Create and Submit a robots.txt File | Google Search Central  |  Documentation  |  Google for Developers

A robots.txt file lives at the root of your site. Learn how to create a robots.txt file, see examples, and explore robots.txt rules.



Bing

Create and Submit a robots.txt File | Google Search Central  |  Documentation  |  Google for Developers

https://developers.google.com/search/docs/crawling-indexing/robots/create-robots-txt

A robots.txt file lives at the root of your site. Learn how to create a robots.txt file, see examples, and explore robots.txt rules.



DuckDuckGo

https://developers.google.com/search/docs/crawling-indexing/robots/create-robots-txt

Create and Submit a robots.txt File | Google Search Central  |  Documentation  |  Google for Developers

A robots.txt file lives at the root of your site. Learn how to create a robots.txt file, see examples, and explore robots.txt rules.

  • General Meta Tags

    11
    • title
      Create and Submit a robots.txt File | Google Search Central  |  Documentation  |  Google for Developers
    • title
      Google
    • title
      Google
    • google-signin-client-id
      721724668570-nbkv1cfusk7kk4eni4pjvepaus73b13t.apps.googleusercontent.com
    • google-signin-scope
      profile email https://www.googleapis.com/auth/developerprofiles https://www.googleapis.com/auth/developerprofiles.award
  • Open Graph Meta Tags

    9
    • og:site_name
      Google for Developers
    • og:type
      website
    • og:title
      Create and Submit a robots.txt File | Google Search Central  |  Documentation  |  Google for Developers
    • og:description
      A robots.txt file lives at the root of your site. Learn how to create a robots.txt file, see examples, and explore robots.txt rules.
    • og:url
      https://developers.google.com/search/docs/crawling-indexing/robots/create-robots-txt
  • Twitter Meta Tags

    1
    • twitter:card
      summary_large_image
  • Link Tags

    14
    • apple-touch-icon
    • canonical
      https://developers.google.com/search/docs/crawling-indexing/robots/create-robots-txt
    • manifest
      /_pwa/developers/manifest.json
    • preconnect
      //www.gstatic.com
    • preconnect
      //fonts.gstatic.com
  • Website Locales

    20
    • AR country flagar
      https://developers.google.com/search/docs/crawling-indexing/robots/create-robots-txt?hl=ar
    • DE country flagde
      https://developers.google.com/search/docs/crawling-indexing/robots/create-robots-txt?hl=de
    • EN country flagen
      https://developers.google.com/search/docs/crawling-indexing/robots/create-robots-txt
    • ES country flages
      https://developers.google.com/search/docs/crawling-indexing/robots/create-robots-txt?hl=es
    • 419 country flages-419
      https://developers.google.com/search/docs/crawling-indexing/robots/create-robots-txt?hl=es-419

Links

211