fuck off ChatGPT.
This commit is contained in:
parent
407ac62a63
commit
854351610c
|
@ -45,7 +45,7 @@ seeds:
|
||||||
- pinterest*
|
- pinterest*
|
||||||
|
|
||||||
generateWACZ: true
|
generateWACZ: true
|
||||||
text: true%
|
text: true
|
||||||
```
|
```
|
||||||
|
|
||||||
What happens behind the scenes is a browser that's fired up and controlled by Puppeteer, where requests, responses, and resources are recorded and links are followed according to the depth configuration. The exclude regex values don't seem to be working that well, and depending on the size of the website, Docker will be running a _long_ time, but the end result is a single archive that's yours forever!
|
What happens behind the scenes is a browser that's fired up and controlled by Puppeteer, where requests, responses, and resources are recorded and links are followed according to the depth configuration. The exclude regex values don't seem to be working that well, and depending on the size of the website, Docker will be running a _long_ time, but the end result is a single archive that's yours forever!
|
||||||
|
|
|
@ -1 +1,15 @@
|
||||||
|
Sitemap: https://brainbaking.com/sitemap.xml
|
||||||
User-agent: *
|
User-agent: *
|
||||||
|
Disallow:
|
||||||
|
|
||||||
|
User-agent: ChatGPT-User
|
||||||
|
Disallow: /
|
||||||
|
|
||||||
|
User-agent: Mediapartners-Google
|
||||||
|
Disallow: /
|
||||||
|
|
||||||
|
User-agent: AdsBot-Google
|
||||||
|
Disallow: /
|
||||||
|
|
||||||
|
User-agent: adidxbot
|
||||||
|
Disallow: /
|
||||||
|
|
Loading…
Reference in New Issue