Added robots.txt

This commit is contained in:
John Crepezzi 2011-11-22 19:59:03 -05:00
parent 5b66695ca6
commit 2299819781
2 changed files with 2 additions and 1 deletions

1
TODO
View file

@ -6,7 +6,6 @@ add link to about page
allow power users to force highlight lang - by supplying an extension!!!!!!
maybe start serving highlighting on the other end to save transfer on highlight.js
start using CDNs for most assets
robots.txt nofollow
kick expiration back by increment on each view
# shared version only

2
static/robots.txt Normal file
View file

@ -0,0 +1,2 @@
User-agent: *
Disallow: /