this post was submitted on 20 Nov 2023
27 points (93.5% liked)

Selfhosted

40329 readers
426 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

  1. Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.

  2. No spam posting.

  3. Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.

  4. Don't duplicate the full text of your blog or github here. Just post the link for folks to click.

  5. Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).

  6. No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 1 year ago
MODERATORS
 

Hi everyone. I was considering backup options to Glacier Deep Archive, and wanted to know:

  1. Which software do you use to encrypt client-side, obfuscate, compress and deduplicate the data before you send it to S3?
  2. What is the difference between Restore Requests (bulk) and Outbound data transfer and which one will I be using when I want to pull my data from AWS?

I'll be storing approximately 8TB or so of data, which is why I was looking at inexpensive ways to back it up other than buying an HDD outright.

Thanks!

you are viewing a single comment's thread
view the rest of the comments
[–] MigratingtoLemmy@lemmy.world 1 points 1 year ago (1 children)

Thanks, I do know of rclone and intend to study it. I was just wondering if the likes of borg, duplicati etc could be used.

[–] lal309@lemmy.world 1 points 1 year ago (1 children)

Can’t speak for those but I tried Kopia and it did the job okay. Ultimately tho I landed on rclone.

[–] MigratingtoLemmy@lemmy.world 1 points 1 year ago (1 children)

Which cloud provider do you use?

[–] lal309@lemmy.world 1 points 1 year ago (1 children)

I’m currently using Backblaze. I also researched Wasabi and AWS.

[–] MigratingtoLemmy@lemmy.world 1 points 1 year ago (1 children)

How much of storage do you use on B2? Does it not feel quite expensive to you? Even Wasabi is quite expensive, although it's not as bad as AWS.

I was recommended iDrive e2 by another commentor, and now that I look at it, it is likely the best product I have come across other than factoring in reliability. I have never heard of this company before, and considering that this is very important data to me, I'd like to have a reliable company behind it.

[–] lal309@lemmy.world 2 points 1 year ago (1 children)

Well here’s my very abbreviated conclusion (provided I remember the details appropriately) when I did the research about 3 months ago.

Wasabi - okay pricing, reliable, s3 compatible, no charges to retrieve my data, pay for 1tb blocks (wasn’t a fan of this one), penalty for data retrieval prior to a “vesting” period (if I remember correctly, you had to leave the data there for 90 days before you could retrieve it at no cost. Also not a big fan of this one)

AWS - I’m very familiar with it due to my job, pricing is largely influenced by access requirements (how often and how fast do I want to retrieve my data), very reliable, s3, charges for everything (list, read, retrieve, etc). This is the real killer and largely unaccounted cost of AWS.

Backblaze - okay pricing, reliable, s3 compliant, free retrieval of data up to the same amount that you store with them (read below), pay by the gig (much more flexible than Wasabi). My heartburn with Backblaze was that retrieval stipulation. However, they have recently increased it to free up to 3x of what you store with them which is super awesome and made my heartburn go away really quickly.

I actually chose Backblaze before the retrieval policy change and it has been rock solid from the start. Works seamlessly with the vast majority of utilities that can leverage s3 compliant storage. Pricing wise, I honestly don’t think it’s that bad

Hope this helps

[–] MigratingtoLemmy@lemmy.world 1 points 1 year ago* (last edited 1 year ago) (1 children)

Here's my situation; I anticipate about 8TB that I will need to store reliably.

That's $50 with BackBlaze B2 a month.

I can get 2 12TB drives for $500 total, and keep one/both of them in remote locations (may or may not be connected to the internet, so I suppose the convenience just isn't there like the Cloud).

The supposed value of the cloud is becoming a bit difficult for me to justify TBH. No wonder B2 is reliable, but if I have 2 drives acting as cold storage in different locations (I will be encrypting the contents), is that a better idea than Cloud storage/BackBlaze specifically? I have been assured that the remote locations should be fine for the most part, other than for natural calamities.

[–] lal309@lemmy.world 2 points 1 year ago (1 children)

Honestly what really matters (imo) is that you do offsite storage. Cloud, a friends house, your parents, your buddy’s NAS, whatever. Just get your data away from your “production/main” site.

For me, I chose cloud for two main reason. First, convenience. I could use a tool to automate the process of moving data offsite in a reliable manner thus keeping my offsite backups almost identical to my main array and easy retrieval should I need it. Second, I don’t really have family or friends nearby and/or with the hardware to support my need for offsite storage.

There are lots of pros and cons of each, let alone add your specific needs and circumstances on top of it.

If you can use the additional drives later on in your main array, some other server or a different purpose then it may be worth while exploring the drives (my concern would be ease of keeping offsite data up to par with main data). If you don’t like it for one reason or the other, you can always repurpose the drives and give cloud storage a try. Again, the important thing is to do it in the first place (and encrypt it client side).

[–] MigratingtoLemmy@lemmy.world 1 points 1 year ago

There are 3 main reasons (in my particular scenario) that might prompt me to go for the cloud:

  1. Reliability of infrastructure.
  2. Convenience.
  3. (Supposed) Bitrot protection (I won't have the protection, just the detection, since I'll be using standalone drives with ZFS).

I need to think a bit more. Thanks!