cross-posted from: https://lemmy.ml/post/9347983

Make sure you subscribe to !announcements@lemmy.ml

What is Lemmy?

Lemmy is a self-hosted social link aggregation and discussion platform. It is completely free and open, and not controlled by any company. This means that there is no advertising, tracking, or secret algorithms. Content is organized into communities, so it is easy to subscribe to topics that you are interested in, and ignore others. Voting is used to bring the most interesting items to the top.

Major Changes

This release is very large with almost 400 commits since 0.18.5. As such we can only give a general overview of the major changes in this post, and without going into detail. For more information, read the full changelog and linked issues at the bottom of this post.

Improved Post Ranking

There is a new scaled sort which takes into account the number of active users in a community, and boosts posts from less-active communities to the top. Additionally there is a new controversial sort which brings posts and comments to the top that have similar amounts of upvotes and downvotes. Lemmy's sorts are detailed here.

Instance Blocks for Users

Users can now block instances. Similar to community blocks, it means that any posts from communities which are hosted on that instance are hidden. However the block doesn't affect users from the blocked instance, their posts and comments can still be seen normally in other communities.

Two-Factor-Auth Rework

Previously 2FA was enabled in a single step which made it easy to lock yourself out. This is now fixed by using a two-step process, where the secret is generated first, and then 2FA is enabled by entering a valid 2FA token. It also fixes the problem where 2FA can be disabled without passing any 2FA token. As part of this change, 2FA is disabled for all users. This allows users who are locked out to get into their account again.

New Federation Queue

Outgoing federation actions are processed through a new persistent queue. This means that actions don't get lost if Lemmy is restarted. It is also much more performant, with separate senders for each target instance. This avoids problems when instances are unreachable. Additionally it supports horizontal scaling across different servers. The endpoint /api/v3/federated_instances contains details about federation state of each remote instance.

Remote Follow

Another new feature is support for remote follow. When browsing another instance where you don't have an account, you can click the subscribe button and enter the domain of your home instance in the popup dialog. It will automatically redirect you to your home instance where it fetches the community and presents a subscribe button. Here is a video showing how it works.

Authentication via Header or Cookie

Previous Lemmy versions used to send authentication tokens as part of the parameters. This was a leftover from websocket, which doesn't have any separate fields for this purpose. Now that we are using HTTP, authentication can finally be passed via jwt cookie or via header Authorization: Bearer . The old authentication method is not supported anymore to simplify maintenance. A major benefit of this change is that Lemmy can now send cache-control headers depending on authentication state. API responses with login have cache-control: private, those without have cache-control: public, max-age=60. This means that responses can be cached in Nginx which reduces server load.

Moderation

Reports are now resolved automatically when the associated post/comment is marked as deleted. This reduces the amount of work for moderators. There is a new log for image uploads which stores uploader. For now it is used to delete all user uploads when an account is purged. Later the list can be used for other purposes and made available through the API.

Cursor based pagination

0.19 adds support for cursor based pagination on the /api/v3/post/list endpoint. This is more efficient for the database. Instead of a query parameter ?page=3, listing responses now include a field "next_page": "Pa46c" which needs to be passed as ?page_cursor=Pa46c. The existing pagination method is still supported for backwards compatibility, but will be removed in the next version.

User data export/import

Users can now export their data (community follows, blocklists, profile settings), and import it again on another instance. This can be used for account migrations and also as a form of backup. The export format is designed to remain unchanged for a long time. You can make regular exports, and if the instance becomes unavailable, register a new account and import the data. This way you can continue using Lemmy seamlessly.

Time zone handling

Lemmy didn't have any support for timezones, which led to bugs when federating with other platforms. This is now fixed by using UTC timezone for all timestamps.

ARM64 Support

Thanks to help from @raskyld and @kroese, there are now offical Lemmy releases for ARM64 available.

Activity now includes voters

• Previously, site and community activity counts were only based on people who commented, or posted. Those counts now include anyone who voted on a comment or post as well. Thanks to @Ategon for this change.

Upgrade instructions

Follow the upgrade instructions for ansible or docker. The upgrade should take less than 30 minutes.

If you need help with the upgrade, you can ask in our support forum or on the Matrix Chat.

Pict-rs 0.5 is also close to releasing. The upgrade takes a while due to a database migration, so read the migration guide to speed it up. Note that Lemmy 0.19 still works perfectly with pict-rs 0.4.

Thanks to everyone

We'd like to thank our many contributors and users of Lemmy for coding, translating, testing, and helping find and fix bugs. We're glad many people find it useful and enjoyable enough to contribute.

Support development

We (@dessalines and @nutomic) have been working full-time on Lemmy for over three years. This is largely thanks to support from NLnet foundation, as well as donations from individual users.

This month we are running a funding drive with the goal of increasing recurring donations from currently €4.000 to at least €12.000. With this amount @dessalines and @nutomic can each receive a yearly salary of €50.000 which is in line with median developer salaries. It will also allow one additional developer to work fulltime on Lemmy and speed up development.

Read more details in the funding drive announcement.

cross-posted from: https://lemmy.ml/post/9347983

What is Lemmy?

Lemmy is a self-hosted social link aggregation and discussion platform. It is completely free and open, and not controlled by any company. This means that there is no advertising, tracking, or secret algorithms. Content is organized into communities, so it is easy to subscribe to topics that you are interested in, and ignore others. Voting is used to bring the most interesting items to the top.

Major Changes

This release is very large with almost 400 commits since 0.18.5. As such we can only give a general overview of the major changes in this post, and without going into detail. For more information, read the full changelog and linked issues at the bottom of this post.

Improved Post Ranking

There is a new scaled sort which takes into account the number of active users in a community, and boosts posts from less-active communities to the top. Additionally there is a new controversial sort which brings posts and comments to the top that have similar amounts of upvotes and downvotes. Lemmy's sorts are detailed here.

Instance Blocks for Users

Users can now block instances. Similar to community blocks, it means that any posts from communities which are hosted on that instance are hidden. However the block doesn't affect users from the blocked instance, their posts and comments can still be seen normally in other communities.

Two-Factor-Auth Rework

Previously 2FA was enabled in a single step which made it easy to lock yourself out. This is now fixed by using a two-step process, where the secret is generated first, and then 2FA is enabled by entering a valid 2FA token. It also fixes the problem where 2FA can be disabled without passing any 2FA token. As part of this change, 2FA is disabled for all users. This allows users who are locked out to get into their account again.

New Federation Queue

Outgoing federation actions are processed through a new persistent queue. This means that actions don't get lost if Lemmy is restarted. It is also much more performant, with separate senders for each target instance. This avoids problems when instances are unreachable. Additionally it supports horizontal scaling across different servers. The endpoint /api/v3/federated_instances contains details about federation state of each remote instance.

Remote Follow

Another new feature is support for remote follow. When browsing another instance where you don't have an account, you can click the subscribe button and enter the domain of your home instance in the popup dialog. It will automatically redirect you to your home instance where it fetches the community and presents a subscribe button. Here is a video showing how it works.

Authentication via Header or Cookie

Previous Lemmy versions used to send authentication tokens as part of the parameters. This was a leftover from websocket, which doesn't have any separate fields for this purpose. Now that we are using HTTP, authentication can finally be passed via jwt cookie or via header Authorization: Bearer <jwt>. The old authentication method is not supported anymore to simplify maintenance. A major benefit of this change is that Lemmy can now send cache-control headers depending on authentication state. API responses with login have cache-control: private, those without have cache-control: public, max-age=60. This means that responses can be cached in Nginx which reduces server load.

Moderation

Reports are now resolved automatically when the associated post/comment is marked as deleted. This reduces the amount of work for moderators. There is a new log for image uploads which stores uploader. For now it is used to delete all user uploads when an account is purged. Later the list can be used for other purposes and made available through the API.

Cursor based pagination

0.19 adds support for cursor based pagination on the /api/v3/post/list endpoint. This is more efficient for the database. Instead of a query parameter ?page=3, listing responses now include a field "next_page": "Pa46c" which needs to be passed as ?page_cursor=Pa46c. The existing pagination method is still supported for backwards compatibility, but will be removed in the next version.

User data export/import

Users can now export their data (community follows, blocklists, profile settings), and import it again on another instance. This can be used for account migrations and also as a form of backup. The export format is designed to remain unchanged for a long time. You can make regular exports, and if the instance becomes unavailable, register a new account and import the data. This way you can continue using Lemmy seamlessly.

Time zone handling

Lemmy didn't have any support for timezones, which led to bugs when federating with other platforms. This is now fixed by using UTC timezone for all timestamps.

ARM64 Support

Thanks to help from @raskyld and @kroese, there are now offical Lemmy releases for ARM64 available.

Activity now includes voters

• Previously, site and community activity counts were only based on people who commented, or posted. Those counts now include anyone who voted on a comment or post as well. Thanks to @Ategon for this change.

Upgrade instructions

Follow the upgrade instructions for ansible or docker. The upgrade should take less than 30 minutes.

If you need help with the upgrade, you can ask in our support forum or on the Matrix Chat.

Pict-rs 0.5 is also close to releasing. The upgrade takes a while due to a database migration, so read the migration guide to speed it up. Note that Lemmy 0.19 still works perfectly with pict-rs 0.4.

Thanks to everyone

We'd like to thank our many contributors and users of Lemmy for coding, translating, testing, and helping find and fix bugs. We're glad many people find it useful and enjoyable enough to contribute.

Support development

We (@dessalines and @nutomic) have been working full-time on Lemmy for over three years. This is largely thanks to support from NLnet foundation, as well as donations from individual users.

This month we are running a funding drive with the goal of increasing recurring donations from currently €4.000 to at least €12.000. With this amount @dessalines and @nutomic can each receive a yearly salary of €50.000 which is in line with median developer salaries. It will also allow one additional developer to work fulltime on Lemmy and speed up development.

Read more details in the funding drive announcement.

On Reddit, when a comment is removed or deleted, it still shows up in the comment count. You will see an interesting post with comments, but when you open it you'll see that it's actually empty. This is probably done for shadowban reasons, like the vote fuzzing in the other post?

On Lemmy the count gets updated. I like that

So I’m developing a mobile client using React Native, where I’m utilizing Lemmy’s messaging functionality as well. This makes it extremely crucial to have notification support (including push notifications).

How are you guys dealing with this problem? This is what I think an elegant solution could look like. We would need to achieve two things:

1. Bring back websockets only for notifications by directly changing lemmy server side code.
2. Find where the email notification code is at, and simply implement expo notifications there.

Whaddya think?

When making a post on Lemmy, it automatically does searches for similar existing posts. On Reddit you often have the issue of people asking the same questions over and over, this feature should definitely help reduce that!

We can follow lemmy accounts from Mastodon but I was wondering if we can do it the other way round?

I tried searching the username of some Masodon accounts into the search bar on Lemmy and nothing came up but when I search up the username of any Lemmy account, they do show up.

So, I'm guessing it's not yet possible?

For example, I'm interested in programming.dev. I want to see what communities they have and how their local posts look like. Therefore, I thought it would be a nice idea for there to be a button which you can press to just see posts from that instance. This would be the same as simply navigating to that instance's homepage, but with the added bonus that you are logged in and can interact with posts.

Another interesting idea would be a place to view a list of communities local to that instance. For example, a "sort by instance" button could be added to the community tab, where instead of only showing local and all communities, you can also view all communities local to an instance (such as all communities on programming.dev.

What are your thoughts?

When I was trying to subscribe to a news community from another instance of lemmy, the button read "Subscribe Pending" What does this mean? Does it mean that they have to approve me before I subscribe, or does it mean that the two instances have to communicate with each other, or something else?

tl;dr: to reduce federation api calls and to reduce issues with defederation, maybe some instances should only be for communities with no user signups, and some instances should be only for user signups with no communities (you would have to make a post or PM to request the admin to make a community for you)

cross-posted from: https://fanaticus.social/post/265339

DRAFT Work in Progress - Updates will be noted in the comments.

I find I have been a bit repetitive in different threads talking about Lemmy / Kbin (collectively, the Threadiverse), so I thought I would put my thoughts together in one place where I can cover it in detail, and revise my thoughts as they evolve. So, here it is...

The Problems

Why can't we merge / sync all the communities with the same name?

This illustrates a fundamental misunderstanding of how Lemmy and ActivityPub work. It implies that local communities are somehow better than federated communities, and that synchronizing different communities would somehow be better / more efficient than just subscribing to the federated community. That's just plain wrong.

Once a community is federated, accessing and interacting with the community is exactly the same as for a local community. The content is exactly the same, and changes are automatically shared among subscribing servers.

The real problem is every instance wanting to be the instance for Reddit knock-off communities. I won't deny that there are significant financial and ego reasons why admins want to accomplish this end. However, this is not the best approach for Lemmy.

The admins of my instance are doing bad things!

Folks, admins need to admin. Each instance is going to have its own policies driven by their personal values and by the legalities of where the server is hosted.

I want to host an instance, but the storage & network requirements are too high

This is a genuine concern - there are two things fundamental to Lemmy that cause this:

1. Each instance needs to keep a complete copy of every community that any user on the instance subscribes to. The storage overhead per user is especially high on instances with not a lot of users.
2. Each community has to share its changes with every instance that has subscribed to it. So when a user on instance A makes a post to a community on instance B, A sends that info to B, then B must send a copy of that post to every other instance with subscribers.

The Solution

Communities

Communities should be spread out across multiple instances, with a small number of like-minded communities on each instance. An example of something like this this would be Discord servers with multiple channels.

• Users on community focused instances should be limited to admins and mods. These should not be primary browsing accounts.
• Community instances can be much more restrictive with their login & firewall policies, making these more secure. Improved remote moderation could limit logins to admins, so the UI itself could be firewalled.
• Businesses, News Media, Celebrities, etc., should host their own community instances so that they can protect their brand and not be subject to third party content policies. Further, instances which are not compatible with the brand's image can be defederated without disrupting the brand's online presence.

Users

Users should congregate on user focused instances.

• Local communities on user instances should be limited to meta topics and possibly a few broad general interest communities.
• User instances can serve as a cache for the distributed network of communities, limiting the duplication of content.
• User instances can be hardened for user facing security

How does this address the problems

Storage & Network Requirements

Having users concentrate on user instances reduces the storage overhead per user, because if multiple users on an instance subscribe to the same communities, there is still only one copy of the community for the instance.

On the network side of things, this reduces the amount of redistribution required by the community instance, because there would be fewer user instances to host subscribers.

In summary, the approach of split user & community instances is really optimal for ActivityPub, because user instances effectively become cacheing servers for communities. This greatly reduces the cost to host community instances.

randomly found this post, curious what other people think about this approach

this is exactly what I do with https://lemmy.mods4ever.com/

only my admin user is on there and it isn't subscribed to any remote communities, Lemmy is barely using any resources on my server it's basically free

I've actually thought about running 2 separate instances like lemmyusers.mods4ever.com and lemmycommunities.mods4ever.com or something like that

originally posted by @cerevant@lemmy.world aka @cerevant@fanaticus.social aka @cerevant@lemm.ee (according to their profile)

I hope that this post doesn't come across as impatient, or demanding -- I fully understand that Lemmy is donation funded, and is created out of the benevolence of the devs. I am merely curious if any announcement has been made in regards to a specific ETA. I am certainly excited for its release, but I also completely understand that there are large restrictions on the developing resources for Lemmy.

Disclaimer: I like the Fediverse, Lemmy, and the concept of federation, I've been here for two years, and I feel grateful towards people working on this platform - devs and admins and mods and everyone else. As such, I hope that what I'm voicing is interpreted as constructive criticism and food for discussion.

TL;DR: I'll list some issues with Lemmy, how they relate to Reddit, and a few proposals on what should be done to address them.

The issues

When you're posting/commenting you're supposed to acknowledge and follow up to three independent sets of rules: of the comm, of the comm's instance, and of your instance. This is a burden for good users, and yet another excuse for bad users to ignore the rules.

There are also up to three groups of rule enforcers, in any situation: two admin teams and a mod team. If any of those goes rogue (greedy pigboy or powerjanny style), you got a problem.

Usually the ones enforcing the rules - the mods - are the group that, by design, lacks access to user info like IPs. So they either play whack-a-mole with old trolls under new accounts, or they rely on assumptions (i.e. stupidity) to keep control of their comms.

Your feed depends on which instances yours is federated with. So you either deal with the fact that you won't get content that you'd otherwise want, or you register into multiple instances to check multiple, partially overlapping feeds. One by one.

Federated instances mirroring content from each other causes sync issues (got removed from A, but not B? You'll still see it in B), storage issues (raising the requirements for people to create their own instances), and it's a big liability (cue to CP being posted to LW, and every single admin team removing it from their own instances).

The biggest instance (by MAU) is as large as the seven following instances combined. This sort of demographic concentration is bound to defeat the advantages of a federation (sharing the burden, sharing the power) without alleviating its cons (added complexity).

The top 10 instances is mostly populated by general purpose instances, doing redundant efforts to provide the same content to the users.

What do those issues have to do with each other?

Look at Reddit.

• Users want their own Reddit communities, but they can't build new "Reddit instances". So they create their communities as "vassals" (subreddits) of the single Reddit instance.
• Since you always post in the same Reddit instance as you registered to, there are no federation woes like "I want content from instance A, but I'm in instance B and they don't federate", or "admins of my instance vs. admins of the instance where I'm posting".
• Reddit cannot rely on other instances to provide content for its users. As such, it hosts all its content in a single, general-purpose instance.

I believe that, once you apply those three aspects of Reddit to a federation, you get the issues that I mentioned.

In other words those issues are born from trying to replicate a non-federation into a federation.

So, what should be done in your opinion?

I'm no coder, nor I want to pretend to be one, and I'm aware that some of those might not be viable. Still, if I had to propose something...

First of all, a change of paradigm: we (users: including mods, admins, developers, everyone) should see Lemmy first and foremost as a federation of forums and advertise it as such. Similarities with Reddit should be only secondary.

People who code in Rust would do an amazing job if they focused on instance creation and management. Ideally, it should be feasible even for a tech-illiterate granny running a potato computer to spin up her own instance.

I think that content mirroring needs to go away, with the users pulling the content straight from the instances where it's created.

Interface developers should expect users to have 2+ accounts, and to log into all their accounts at the same time. The resulting feed should be a combination of the feed of those instances; handle this through the interface/front-end. And when the user is posting/commenting, ideally they should be able to choose which account to use, on a per-community basis.

Desktop users should be encouraged to migrate from "my instance's website" to instance-agnostic front-ends, such as Alexandrite and Slemmy. [This doesn't affect mobile users, I believe.]

We should be contributing more to specific-purpose instances (for example: mander.xyz, ani.social, etc.), at the detriment of general-purpose instances (for example: lemmy.world). Perhaps, at the start even migrate our comms to those instances.

Eventually [in the far, far future] I think that the concept of subreddit-like communities should be deprecated, with communities becoming simple sub-forums of the instance where they're hosted.

By default, admins should focus mostly on the activity inside their own instances. Let the behaviour of their users in other instances up to those admins; a dog with two owners ends either overfed or starved.

When possible/reasonable, admins should be moderating more communities in their own instances.

I think the title says it all. Basically, when a new comment appears on your targeted post, it sends you a PM about it.

@PostWatchBot@lemy.lol

Usage

• Subscribe to a post: just mention the bot in the comments or send the link to the bot via PM.
• Unsubscribe from a post: send PM to the bot with stop text and link of the post like stop https://lemmy.ml/post/1234
• Unsubscribe completely: Send PM to the bot and add stop text to your message. It will unsubscribe you from all subscriptions.

Note: the bot sends only one notification per post. It waits for the previous notification to be marked as read for new comments.

Made with @CannotSleep420@hexbear.net's lemmy-bot project 🙏 Tomorrow I will publish the code publicly after adding README and self-hosting guide.

Makes me happy. It's easy to have a good time here.

0.19 brings many improvements but "Adding a scaled sort, to boost smaller communities" is the one I'm most excited about.

you can see it now on lemmy.ml (no account required) here and you'll almost certainly find communities you didn't know existed which you want to subscribe to.

That's all I ask for 0.19.0

Only seen it the past hour or so but I’ve already set it as my default. It seems to work exactly as I’d hoped where smaller but no less interesting communities get their posts higher in my feed.

This could be a huge change and I think lemmy would benefit from those of us caring about the idea being vocal about it and assessing its efficacy.

Noticed I was logged out of lemmy.ml this morning. When I logged in, everything looked the same, but... "All" loaded instantly. Switching to "Subscribed" was just as fast. Post thumbnails came up as quickly as I could scroll.

I don't know if it's the new software or if y'all cleared out some cruft when restarting the services, but from this end-user's perspective, Lemmy 0.19.0-rc.8 flies. Nicely done!

I love that folks use these archive.org links like this one here:

https://web.archive.org/web/20231125030342/https://www.kansascity.com/news/local/crime/article281901303.html

How are they finding these or making them? Let’s say I’m reading NYTimes or something, what do I do to link to the same article but archived?

Does Lemmy have anyway to pre-fill the Create Post details from the URL? For example Reddit allows for: https://www.reddit.com/submit?url=https%3A%2F%2Fstackoverflow.com%2Fquestions%2F24823114%2Fpost-to-reddit-via-url&title=Post%20to%20Reddit%20via%20URL.

I'm aware that this gets slightly tricky across instances (due to different domains), however I was hoping to build a system where the user enters their Lemmy domain, and it takes them to a page so they can create a post about a given URL.

Any ideas?

@lemmy ban with no citation?

the fact that @lemmy bans you tells you everything you need to know about it as a platform. I didn't want to be accusatory when I thought my comments were being deleted but it seems that they were.

cross-posted from: https://lemmy.dbzer0.com/post/8874717

I have just deployed a script and a mastodon bot which attempt to hashtag lemmy posts so that they are better discoverable in microblogging services.

Please check the README for the why and the how.

If you have a microblogging account, please consider following the bot account which will help its hashtags federate to your instance's public timeline.

Many thanks to @jgrim@discuss.online for hosting the bot.

PS: If you have a mastodon account, you can reply to your posts on mastodon (just search for their url) and add hashtags to your replies. This will achieve a quick and dirty version of what this bot is doing

Just sayin’