this post was submitted on 08 Jan 2024
65 points (93.3% liked)

Selfhosted

40394 readers
379 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

  1. Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.

  2. No spam posting.

  3. Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.

  4. Don't duplicate the full text of your blog or github here. Just post the link for folks to click.

  5. Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).

  6. No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 2 years ago
MODERATORS
65
submitted 10 months ago* (last edited 10 months ago) by jaykay@lemmy.zip to c/selfhosted@lemmy.world
 

Hi guys! I think I'm over Joplin. Don't get me wrong, it's simple, it works, but… why is it Postgres db…. I have the server on a small box with like 250 GB of space and backing it up with kopia to Backblaze with free 10 GB, so I'm a bit storage cautious.

With each snapshot, it seems like a good chunk of the database changes, even when I don't use Joplin that day. That results in kopia backing up those changed files, and backups keep growing. Right now the Joplin database is like 200Mb, BUT when I export the notes from the app… all of them weigh 2Mb… including images. Yes there is versioning of notes, but they shouldn't be that big after one-two months lol.

I know I know, I'm being a bit weird about it, but I'm getting daily notifications about backups and I see how they grow each day.

Anyway, do you have any alternatives that have an app on iOS and on Linux? Or should I just use Apple Notes in the browser? Thanks

EDIT: The answer was easier than I thought. Just don’t back it up, it’s synced which means each device has a copy of it anyway so there is not really need for it, thanks @vvv@programming.dev !

you are viewing a single comment's thread
view the rest of the comments
[–] atzanteol@sh.itjust.works 1 points 10 months ago (2 children)

I think you need to learn more about how databases work. They don't typically reclaim deleted space automatically for performance reasons. Databases like to write to a single large file they can then index into. Re-writing those files is expensive so left to the DBA (you) to determine when it should be done.

And how are you backing up the database? Just backing up /var/lib/postgres? Or are you doing a pg_dump? If the former then it's possible your backups won't be coherent if you haven't stopped your database and it will contain that full history of deleted stuff. pg_dump would give you just the current data in a way that will apply properly to a new database should you need to restore

You can also consider your backup retention policy. How many backups do you need for how long?

[–] seang96@spgrn.com 2 points 10 months ago

Setup backup hooks with velero and kopia on a HA postres cluster this week. Biggest DB is Lemmy and that shrinks by a factor of 10 using pgdunp with custom archive. Dumping is 100% the way to go!

Similarly I should do this for my sqlite applications, it looks like kopia can't do incremental backups with them and thinking about it, it makes sebse, likely sane reasons you mentioned.

[–] jaykay@lemmy.zip 1 points 10 months ago

You are right, I should. They are a bit more complicated than I anticipated, and apparently I'm doing everything wrong, haha. I have backups set up to go 2 years back, but I'm checking backblaze occasionally to check, so it shouldn't be an issue. I have two months so far lol Thanks for the write-up :)