You can install the package using composer
$ composer require tequilarapido/robotstxt
Only if you are using Laravel below 5.5, you will need to add the Service Provider inside config/app.php
'providers' => [
// Laravel Framework Service Providers...
//...
// Package Service Providers
Tequilarapido\RobotsTxt::class,
],
Finally, you need to publish config file
$ php artisan vendor:publish --provider="Tequilarapido\RobotsTxt\RobotsTxtProvider" --tag=config
There are 3 types of robots.txt file. In your .env file, add:
ROBOTS_TXT_STATUS= #disallow_all|allow_all|custom
User-agent: *
Disallow: /
User-agent: *
You need to add in your .env file :
ROBOTS_TXT_CUSTOM= #What do you want to be written in your robots.txt file