oseta

We all know what search engines do right? yes Search! baby search! they search every where for everything. And what do web site owners do? The do various SEO (search engine optimization) stuff to make the things easy for the search engines so the SE’s will search the directories inside the web sites easily and bring more traffic to the site!. Hm… Ok C.J.! enough of the technical stuff,why are you telling us all this? This is why…
Do you know that there are some directories in web sites that they don’t want the general public to know about? Well, there are! Then what do the site owners do to hide them from us? They bribe the search engines :D Hm…
well what they do is put a text file including the directory names, that should be excluded when searching. So the search engines read that file first and exclude their content in the search results page……
But you know that us geeks can’t stay out of trouble right? Yes! there’s a way you can see what are the hidden directories are! All you got to do is put /robots.txt after the domain name and see what the site owners don’t want you to see :twisted:
How about checking out what Mr.Bill Gates got to hide from us? :D Here you go
http://www.micr*s*ft.c*m/r*b*ts.txt
And what about larry and sergei? yeh!
http://www.g**gle.c*m/r*b*ts.txt