# See https://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file # Login, registration and password pages have nothing to index, and crawlers were a large share of the Rails # time spent rendering /users/sign_in. User-agent: * Disallow: /users/