I recently setup a new Lemmy instance and was surprised when my feed was mostly empty. I've since learned that a key part of Lemmy's federation is based on a user from your instance subscribing to communities on other instances. Only then, will your instance pull in posts from the subscribed community to your "All" feed.
This means that subscribing to new communities is especially important if you're on a young Lemmy instance since it helps to build out everyone's feed on that instance.
I've found discovering new communities to subscribe to on other instances can be difficult. To help me search for new communities I may be interested in, I tried aggregating as much of the Lemmy fediverse together into a single feed by subscribing to the widest range of Lemmy communities possible. This offers a Lemmy feed that's kind of like reddit.com/r/all. If it's interesting to anyone else, you can find the instance here: https://lemmy.directory.
Hopefully this offers another way to find new communities to subscribe to on other instances.
Here's a better description of my understanding on how Lemmy federates communities and why you might be interested in checking out lemmy.directory: https://lemmy.directory/post/34207.
Hope this helps ease the orientation to how Lemmy federation and communities work.
Do instances fully replicate and locally store remote subscribed communities? My understanding is they are still solely hosted on the original instance; subscribing just opens a window to the community by making your instance aware it exists.
To a first approximation yes, they replicate the posts and comments made after the time of subscription. Images aren't replicated, but posts, comments, votes, and mod actions do replicate.
I don't know what you consider "a window" to be, and maybe this is a fuzzy description of some steps of community discovery... but it's definitely not a complete or coherent view of how instances interact through federation.
Thank you for taking the time to answer. I hope you might be willing to clarify a bit more for me. By "window", I meant just... having access to a remote community via an API gateway, I guess.
I was under the impression that if I try to subscribe to a remote community hosted on
lemmy.worldfromvlemmy.net, that is simply registering the URL of that community into some local directory in my instance, not duplicating the entire community contents intovlemmy.net. And then when I view a thread in that remote community, I am just retrieving the thread data from the host server atlemmy.worldstraight to my browser, not loading some local duplicate of the thread fromvlemmy.net. Seems like it would get out of sync quickly if we are all reading separate local copies of the original.So based on your answer, I am still misunderstanding something. What is the purpose of all the duplication then? Is it just for local caching purposes? Does this not needlessly drive up the amount of traffic because each instance is frantically trying to keep up to date with every other instance, rather than just letting each instance handle the requests for its own communities?
Pretty much everything in your summary was wrong. I can't reasonably type out a complete primer on federation here, but in short... When the first user on an instance subscribes to a remote community, the subscribing server tells the community-hosting server "send me future updates about posts, comments, and votes and such for this community". The subscribing server then stores them locally.
Why do this actual replication rather than just an API gateway?
lemmy.mlrecently when it was struggling and frequently down due to overload.lemmy.worldhas commonly has like 4k active users this week. But it only takes one batch of federated replication messages for the community's instance to serve the browse traffic of all those users... then the reads come out oflemmy.world's db. This spreads the browse workload around the federated network.Can federated networks have problems where the replication traffic becomes "too much"? Yes, they must be carefully designed to avoid that problem. And for some apps, it makes single-user instances sometimes anti-efficient as the federation workload for the instance can exceed the browse workload from the user, but for multi-user instances the federation messages are a rounding error compared to the browse work. But it's a problem that federated networks generally weather, and the replication offers benefits that on-balance outweigh the costs.