rebot文件样例

User-agent: *Disallow: /searchDisallow: /sdchDisallow: /groupsDisallow: /imagesDisallow: /catalogsAllow: /catalogs/aboutAllow: /catalogs/p?Disallow: /cataloguesDisallow: /newsAllow: /news/directoryDisallow: /nwshpDisallow: /setnewsprefs?Disallow: /index.html?Disallow: /?Allow: /?hl=Disallow: /?hl=*&Disallow: /addurl/image?Disallow: /pagead/Disallow: /relpage/Disallow: /relcontentDisallow: /imgresDisallow: /imglandingDisallow: /sbdDisallow: /keyword/Disallow: /u/Disallow: /univ/Disallow: /cobrandDisallow: /customDisallow: /advanced_group_searchDisallow: /googlesiteDisallow: /preferencesDisallow: /setprefsDisallow: /swrDisallow: /urlDisallow: /defaultDisallow: /m?Disallow: /m/Disallow: /wml?Disallow: /wml/?Disallow: /wml/search?Disallow: /xhtml?Disallow: /xhtml/?Disallow: /xhtml/search?Disallow: /xml?Disallow: /imode?Disallow: /imode/?Disallow: /imode/search?Disallow: /jsky?Disallow: /jsky/?Disallow: /jsky/search?Disallow: /pda?Disallow: /pda/?Disallow: /pda/search?Disallow: /sprint_xhtmlDisallow: /sprint_wmlDisallow: /pqaDisallow: /palmDisallow: /gwt/Disallow: /purchasesDisallow: /hwsDisallow: /bsd?Disallow: /linux?Disallow: /mac?Disallow: /microsoft?Disallow: /unclesam?Disallow: /answers/search?q=Disallow: /local?Disallow: /local_urlDisallow: /shihui?Disallow: /shihui/Disallow: /froogle?Disallow: /products?Disallow: /products/Disallow: /froogle_Disallow: /product_Disallow: /products_Disallow: /products;Disallow: /printDisallow: /books/Disallow: /bkshp?*q=*Disallow: /books?*q=*Disallow: /books?*output=*Disallow: /books?*pg=*Disallow: /books?*jtp=*Disallow: /books?*jscmd=*Disallow: /books?*buy=*Disallow: /books?*zoom=*Allow: /books?*q=related:*Allow: /books?*q=editions:*Allow: /books?*q=subject:*Allow: /books/aboutAllow: /booksrightsholdersAllow: /books?*zoom=1*Allow: /books?*zoom=5*Disallow: /ebooks/Disallow: /ebooks?*q=*Disallow: /ebooks?*output=*Disallow: /ebooks?*pg=*Disallow: /ebooks?*jscmd=*Disallow: /ebooks?*buy=*Disallow: /ebooks?*zoom=*Allow: /ebooks?*q=related:*Allow: /ebooks?*q=editions:*Allow: /ebooks?*q=subject:*Allow: /ebooks?*zoom=1*Allow: /ebooks?*zoom=5*Disallow: /patents?Disallow: /patents/download/Disallow: /patents/pdf/Disallow: /patents/related/Disallow: /scholarDisallow: /citations?Allow: /citations?user=Disallow: /citations?*cstart=Allow: /citations?view_op=new_profileAllow: /citations?view_op=top_venuesDisallow: /completeDisallow: /s?Disallow: /sponsoredlinksDisallow: /videosearch?Disallow: /videopreview?Disallow: /videoprograminfo?Allow: /maps?hq=http://maps.google.com/help/maps/directions/biking/mapleft.kml&ie=UTF8&ll=37.687624,-122.319717&spn=0.346132,0.727158&z=11&lci=bike&dirflg=b&f=dAllow: /maps?hq=http://maps.google.com/help/maps/phototours/mapleft.xml&h=1100Allow: /maps/api/js?Disallow: /maps?Disallow: /mapstt?Disallow: /mapslt?Disallow: /maps/stk/Disallow: /maps/br?Disallow: /mapabcpoi?Disallow: /maphp?Disallow: /mapprint?Disallow: /maps/api/js/Disallow: /maps/api/staticmap?Disallow: /mld?Disallow: /staticmap?Disallow: /places/Allow: /places/$Disallow: /maps/previewDisallow: /maps/placeDisallow: /help/maps/streetview/partners/welcome/Disallow: /help/maps/indoormaps/partners/Disallow: /lochp?Disallow: /centerDisallow: /ie?Disallow: /sms/demo?Disallow: /katrina?Disallow: /blogsearch?Disallow: /blogsearch/Disallow: /blogsearch_feedsDisallow: /advanced_blog_searchDisallow: /uds/Disallow: /chart?Disallow: /transit?Disallow: /mbd?Disallow: /extern_js/Disallow: /xjs/Disallow: /calendar/feeds/Disallow: /calendar/ical/Disallow: /cl2/feeds/Disallow: /cl2/ical/Disallow: /coop/directoryDisallow: /coop/manageDisallow: /trends?Disallow: /trends/music?Disallow: /trends/hottrends?Disallow: /trends/viz?Disallow: /trends/embed.js?Disallow: /trends/fetchComponent?Disallow: /notebook/search?Disallow: /musicaDisallow: /musicadDisallow: /musicasDisallow: /musiclDisallow: /musicsDisallow: /musicsearchDisallow: /musicspDisallow: /musiclpDisallow: /browsersyncDisallow: /callDisallow: /archivesearch?Disallow: /archivesearch/urlDisallow: /archivesearch/advanced_searchDisallow: /base/reportbadofferDisallow: /urchin_test/Disallow: /movies?Disallow: /codesearch?Disallow: /codesearch/feeds/search?Disallow: /wapsearch?Disallow: /safebrowsingAllow: /safebrowsing/diagnosticAllow: /safebrowsing/report_badware/Allow: /safebrowsing/report_error/Allow: /safebrowsing/report_phish/Disallow: /reviews/search?Disallow: /orkut/albumsAllow: /jsapiDisallow: /views?Disallow: /c/Disallow: /cbkAllow: /cbk?output=tile&cb_client=maps_svDisallow: /recharge/dashboard/carDisallow: /recharge/dashboard/static/Disallow: /translate_a/Disallow: /translate_cDisallow: /translate_fDisallow: /translate_static/Disallow: /translate_suggestionDisallow: /profiles/meAllow: /profilesDisallow: /s2/profiles/meAllow: /s2/profilesAllow: /s2/photosAllow: /s2/staticDisallow: /s2Allow: /s2/search/socialDisallow: /transconsole/portal/Disallow: /gcc/Disallow: /aclkDisallow: /cse?Disallow: /cse/homeDisallow: /cse/panelDisallow: /cse/manageDisallow: /tbproxy/Disallow: /imesync/Disallow: /shenghuo/search?Disallow: /support/forum/search?Disallow: /reviews/polls/Disallow: /hosted/images/Disallow: /ppob/?Disallow: /ppob?Disallow: /adwordsresellersDisallow: /accounts/o8Allow: /accounts/o8/idDisallow: /topicsearch?q=Disallow: /xfx7/Disallow: /squared/apiDisallow: /squared/searchDisallow: /squared/tableDisallow: /toolkit/Allow: /toolkit/*.htmlDisallow: /globalmarketfinder/Allow: /globalmarketfinder/*.htmlDisallow: /qnasearch?Disallow: /app/updatesDisallow: /sidewiki/entry/Disallow: /quality_form?Disallow: /labs/popgadget/searchDisallow: /buzz/postDisallow: /compressiontest/Disallow: /analytics/reporting/Disallow: /analytics/admin/Disallow: /analytics/web/Disallow: /analytics/feeds/Disallow: /analytics/settings/Disallow: /alerts/Disallow: /ads/searchDisallow: /ads/plan/action_plan?Disallow: /ads/plan/api/Disallow: /phone/compare/?Allow: /alerts/manageAllow: /alerts/removeDisallow: /travel/clkDisallow: /hotelfinder/rpcDisallow: /hotels/rpcDisallow: /flights/rpcDisallow: /commercesearch/services/Disallow: /evaluation/Disallow: /chrome/browser/mobile/tourDisallow: /compare/*/apply*Disallow: /forms/perks/Disallow: /baraza/*/searchDisallow: /baraza/*/reportDisallow: /shopping/suppliers/searchDisallow: /ct/Disallow: /edu/cs4hs/Disallow: /trustedstores/s/Disallow: /trustedstores/tm2Disallow: /trustedstores/verifyDisallow: /adwords/proposalDisallow: /shopping/product/Disallow: /shopping/sellerDisallow: /shopping/reviewerDisallow: /about/careers/apply/Disallow: /about/careers/applications/Disallow: /landing/signout.htmlSitemap: http://www.gstatic.com/culturalinstitute/sitemaps/www_google_com_culturalinstitute/sitemap-index.xmlSitemap: http://www.google.com/hostednews/sitemap_index.xmlSitemap: http://www.google.com/maps/views/sitemap.xmlSitemap: http://www.google.com/sitemaps_webmasters.xmlSitemap: http://www.gstatic.com/sitemaps/websearch_hreflang/sitemap_index.xmlSitemap: http://www.google.com/ventures/sitemap_ventures.xmlSitemap: http://www.gstatic.com/dictionary/static/sitemaps/sitemap_index.xmlSitemap: http://www.gstatic.com/earth/gallery/sitemaps/sitemap.xmlSitemap: http://www.gstatic.com/s2/sitemaps/profiles-sitemap.xmlSitemap: http://www.gstatic.com/trends/websites/sitemaps/sitemapindex.xml