this post was submitted on 24 Jul 2024
1 points (100.0% liked)

Technology

59587 readers
2936 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related content.
  3. Be excellent to each another!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, to ask if your bot can be added please contact us.
  9. Check for duplicates before posting, duplicates may be removed

Approved Bots


founded 1 year ago
MODERATORS
 

DuckDuckGo, Bing, Mojeek, and other search engines are not returning full Reddit results any more.

you are viewing a single comment's thread
view the rest of the comments
[–] reddig33@lemmy.world 0 points 4 months ago* (last edited 4 months ago) (1 children)

No, I wouldn’t feel differently. In fact letting search engines scrape and point to your content is what leads people to your site. It’s free advertising. If you’re going to let one search engine in, you should let them all in. If you want to be public, be public. Otherwise put up a login firewall and go private.

[–] capital@lemmy.world 0 points 4 months ago

It's not just search engines. Lots of people on Mastodon were using robots.txt to block ChatGPT (and any other LLM company they knew of) from scraping their sites/blogs.

I disagree, to a point. I want to be able to control my services to the greatest extent possible, including picking who scrapes me.

On the other hand, orgs as large as Google doing this poses a real threat to how the internet works right now which I hate.