User-agent: MJ12bot
User-agent: AhrefsBot
User-agent: BLEXBot
User-agent: Yandex
User-agent: baiduspider
User-agent: SemrushBot
Crawl-delay: 3600