0.04 seconds

Realtime results for robots.txt

0 more results since you started searching. Refresh to see them.

• 
  pageoneresults: Show me a robots.txt file with list of files for Disallow and I'll show you thousands of pages of leakage. Take Disallow paths and do site:
  27 minutes ago from web · Reply · View Tweet

• 
  webseoanalytics: #SEO Tip:Pages blocked in robots.txt continue receiving PageRank when they have links #google #bing #yahoo
  29 minutes ago from API · Reply · View Tweet

• 
  pageoneresults: @TonyVerre Ah, you can't do that. The bot will never make it to the page level directive if Disallowed in robots.txt.
  39 minutes ago from web · Reply · View Tweet · Show ConversationHide Conversation
• 
  TonyVerre: Put the page in Robots.txt and add page-level NOINDEX as well. That should deter any spider from seeing the page. @pageoneresults
  40 minutes ago from TweetDeck · Reply · View Tweet

• 
  pageoneresults: @TonyVerre No, you can't do that. It is one or the other. I never recommend robots.txt, ever. What do you mean both?
  42 minutes ago from web · Reply · View Tweet · Show ConversationHide Conversation
• 
  netbranding: Matt Cutts über Blocken mit robots.txt - SEO.at http://goo.gl/fb/TUXd #seoknowhow
  about 1 hour ago from Google · Reply · View Tweet

• 
  TonyVerre: To have a complete fail safe: Put in ROBOTS.txt (site-wide), and NOINDEX and/or NOFOLLOW at a page-level. @BarryWise @pageoneresults
  about 1 hour ago from TweetDeck · Reply · View Tweet

• 
  pageoneresults: @g1smd You better stop borking my username! :) Oh, I'm well aware of robots.txt perils. Love cleaning them up, BIG boost too. :)
  about 2 hours ago from web · Reply · View Tweet · Show ConversationHide Conversation
• 
  pageoneresults: @g1smd I only use robots.txt for White Listing. Never list any paths in there other than generic one page landers with NO links. :)
  about 2 hours ago from web · Reply · View Tweet · Show ConversationHide Conversation
• 
  g1smd: @SebastianX @PageoneResults Disallow: /whatever in robots.txt still allows URL to appear as URL-only entry in SERPs.
  about 2 hours ago from P3:PeraPeraPrv · Reply · View Tweet · Show ConversationHide Conversation
• 
  topfmodel: RT @seoAT: "Matt Cutts über Blocken mit robots.txt http://bit.ly/99xyPn (expand)"
  about 2 hours ago from PockeTwit · Reply · View Tweet

• 
  cvul: #proxy Why You Should Set Up A robots.txt To Proxy Script http://ping.fm/gjif1
  about 2 hours ago from Ping.fm · Reply · View Tweet

• 
  Pressboxcaller: RT @JulesPolonetsky: RT @SusannahFox: exp date for regrettable online behavior? Have search engine respect robots.txt [180 days] for an item a user wants to age?
  about 2 hours ago from TweetDeck · Reply · View Tweet

• 
  JulesPolonetsky: RT @SusannahFox: exp date for regrettable online behavior? Have search engine respect robots.txt [180 days] for an item a user wants to age?
  about 2 hours ago from TweetDeck · Reply · View Tweet

• 
  mi6friend4all: Should duplicate pages should be blocked by using robots.txt? http://www.youtube.com/watch?v=CJMFYpYQZ0c
  about 2 hours ago from web · Reply · View Tweet