this post was submitted on 10 Jun 2023

4 points (100.0% liked)

Lemmy

12576 readers

2 users here now

Everything about Lemmy; bugs, gripes, praises, and advocacy.

For discussion about the lemmy.ml instance, go to !meta@lemmy.ml.

founded 4 years ago

MODERATORS

nutomic@lemmy.ml

I wrote a bot to import reddit posts from Lemmy to help with the transition. Is this allowed? (lemmy.one)

submitted 1 year ago* (last edited 1 year ago) by tubbadu@lemmy.one to c/lemmy@lemmy.ml

11 comments fedilink hide all child comments

Hello! I wrote a simple bot that periodically checks for new reddit posts and posts them to lemmy, so that people migrating from reddit to lemmy can still be able to see their favourite posts, but familiarizing with lemmy.

currently the coments are not synced, but this may change in the future (perhaps)

Yes, it uses the Reddit API, so it will stop working on the 1st of July, but I think that then I can implement a sort of web scraper to access Reddit posts without the official API, so this may eventually keep working for a while.

this script is currently on my laptop so it will be offline most of the time, but if I get the approval I may host it somewhere to get it running 24h/24.

now the question... Is this allowed? having this bot running 24h/24 on large subreddits will mean a very high quantity of posts. will this cause any problem to Lemmy?

if you want a preview check out https://enterprise.lemmy.ml/c/reddit_memes, where I started syncing a few posts from r/memes

let me know your opinion on this!

==== EDIT

here's the bot source code

The bot is now running in https://sh.itjust.works/c/reddit_memes, let's try to see if it work (I hope that shit just works)

I'm a bit concerned about the legality of this, if anyone has any info please tell me!

top 11 comments

sorted by: hot top controversial new old

[–] usernotfound@lemmy.ml 3 points 1 year ago

Ha! I have been working on the same thing this weekend, except it uses the rss feed for posts and scrapes old.reddit.com for the details. It's written in python, but not quite finished - scraping works, automation not yet.

My plan was to have a separate Lemmy instance for this, where people can also request for new subs to be included. This would reduce the spam in bigger communities, and allow instances to block it all together if they wanted to.

Beside that, I'd pre- or postfix each post with a message it's a copy and a link to the original for copyright reasons. Moderation would be a separate story - Not particularly looking forward to that. Could make it so that if a post were flagged, it would re-aync with the original. Let reddit do the moderation :D

[–] Sibbo@sopuli.xyz 1 points 1 year ago (1 children)

This may be against reddits TOS and may have legal consequences. But I guess mostly for the person operating the bot, and not the Lemmy instance, as long as they block the bot on request.

[–] tubbadu@lemmy.one 0 points 1 year ago (1 children)

Do you think this may be "dangerous" for me?

[–] Sibbo@sopuli.xyz 1 points 1 year ago (1 children)

I have no idea, but at least in EU you may be violating copyright by copying someone else's database (i.e. collection of data). I am not a lawyer though...

[–] tubbadu@lemmy.one 1 points 1 year ago (1 children)

I Read the terms of service and what I understood is that the intellectual property of the content is the users', not reddit's, but I asked on r/legaladvices for security (there is no equivalent community on lemmy yet)

[–] Sibbo@sopuli.xyz 1 points 1 year ago

Yeah database protection is even for stuff that you don't own. You have then spent effort to compile the database. Which in this case is the collection of people's posts. But maybe asks lawyer if that applies in this case.

[–] Die4Ever@programming.dev 1 points 1 year ago* (last edited 1 year ago)

for some reason I still can't find this community from my instance, neither searching method works, and I waited hours and tried again and still nothing

https://programming.dev/search/q/https%3A%2F%2Fenterprise.lemmy.ml%2Fc%2Freddit_memes/type/All/sort/TopAll/listing_type/All/community_id/0/creator_id/0/page/1

https://programming.dev/search/q/!reddit_memes%40enterprise.lemmy.ml/type/All/sort/TopAll/listing_type/All/community_id/0/creator_id/0/page/1

I made a post about it here so maybe it can be debugged https://programming.dev/post/14560

[–] wiki_me@lemmy.ml 1 points 1 year ago

IANAL but i am pretty sure it is illlegal, copyrights prevents copying and iirc reddit users retain copyrights to what they write and you don't have their approval (in the form of a Term of service) to copy it.

Maybe you could automate that approval (e.g. users could have a pinned post on their profit with a copied message).

There are some non profits for open source that might have full time lawyers , maybe you can contact them.

[–] Die4Ever@programming.dev 0 points 1 year ago (1 children)

maybe you should open source the bot?

it could also have filtering rules like only import posts with a minimum score or only the front page of the sub, could be configurable for each subreddit separately

[–] tubbadu@lemmy.one 0 points 1 year ago (1 children)

Yeah I'm literally creating the github repo right now lmao

[–] Die4Ever@programming.dev 1 points 1 year ago

another tool that might be good, idk if it already exists, but an RSS feed to Lemmy importer

actually Reddit supports RSS output, so you might already be using that?