this post was submitted on 10 Nov 2024
84 points (98.8% liked)

Selfhosted

40382 readers
373 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

  1. Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.

  2. No spam posting.

  3. Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.

  4. Don't duplicate the full text of your blog or github here. Just post the link for folks to click.

  5. Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).

  6. No trolling.

Resources:

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 2 years ago
MODERATORS
 

If you think this post would be better suited in a different community, please let me know.


Topics could include (this list is not intending to be exhaustive — if you think something is relevant, then please don't hesitate to share it):

  • Moderation
  • Handling of illegal content
  • Server structure (system requirements, configs, layouts, etc.)
  • Community transparency/communication
  • Server maintenance (updates, scaling, etc.)

Cross-posts

  1. https://sh.itjust.works/post/27913098
you are viewing a single comment's thread
view the rest of the comments
[–] Kalcifer@sh.itjust.works 12 points 2 weeks ago (1 children)

Run the software that scans images for CSAM.

Which software is that?

[–] walden 16 points 2 weeks ago (2 children)

It's called Lemmy-Safety of Fedi-Safety depending on where you look.

One thing to note, I wasn't able to get it running on a VPS because it requires some sort of GPU.

[–] Kalcifer@sh.itjust.works 10 points 2 weeks ago (2 children)

One thing to note, I wasn’t able to get it running on a VPS because it requires some sort of GPU.

This is good to know. I know that you can get a VPS with a GPU, but they're usually rather pricey. I wonder if there's one where the GPU's are shared, and you only get billed by how much the GPU is used. So if there is an image upload, the GPU would kick on to check it, you get billed for that GPU time, then it turns off and waits for the next image upload.

[–] pe1uca@lemmy.pe1uca.dev 5 points 2 weeks ago (2 children)

I don't think there are services like that, since usually this means deploying and destructing an instance, which takes a few minutes (if you just turn off the instance you still get billed).
Probably the best option would be to have a snapshot, which costs way less than the actual instance, and create from it each day or so yo run on the images since it was last destroyed.

This is kind of what I do with my media collection, I process it on my main machine with a GPU, and then just serve it from a low-power one with Jellyfin.

[–] Kalcifer@sh.itjust.works 2 points 2 weeks ago* (last edited 2 weeks ago) (1 children)

create from it each day or so yo run on the images since it was last destroyed.

Unfortunately, for this usecase, the GPU needs to be accessible in real time; there is a 10 second window when an image is posted for it to be processed [1].

References

  1. "I just developed and deployed the first real-time protection for lemmy against CSAM!". @db0@lemmy.dbzer0.com. !div0@lemmy.dbzer0.com. Divisions by zero. Published: 2023-09-20T08:38:09Z. Accessed: 2024-11-12T01:28Z. https://lemmy.dbzer0.com/post/4500908.
    • §"For lemmy admins:"

      [...]

      • fedi-safety must run on a system with GPU. The reason for this is that lemmy provides just a 10-seconds grace period for each upload before it times out the upload regardless of the results. [1]

      [...]

[–] db0@lemmy.dbzer0.com 3 points 2 weeks ago* (last edited 2 weeks ago) (1 children)

You can actually run it in async model without pictrs safety and just have it scan your newly uploaded images directly from storage. It just doesn't prevent upload this way, just deletes them.

[–] Kalcifer@sh.itjust.works 1 points 1 week ago (1 children)

You're referring to using only fedi-safety instead of pictrs-safety, as was mentioned in §"For other fediverse software admins", here, right?

[–] Kalcifer@sh.itjust.works 1 points 2 weeks ago

Probably the best option would be to have a snapshot

Could you point me towards some documentation so that I can look into exactly what you mean by this? I'm not sure I understand the exact procedure that you are describing.

[–] db0@lemmy.dbzer0.com 1 points 2 weeks ago (1 children)

The software is setup in such a way that you can run it on your pc if you have a local gpu. It only needs like 2 gb vram

[–] Kalcifer@sh.itjust.works 1 points 2 weeks ago* (last edited 2 weeks ago) (1 children)

That is a cool feature, but that would mean that all of the web traffic would get returned to my local network (assuming that the server is set up on a remote VPS), which I really don't want to have happen. There is also the added downtime potential cause by the added point of failure of the GPU being hosted in a much more volatile environment (ie not, for example, a tier 3 data center).

[–] db0@lemmy.dbzer0.com 2 points 2 weeks ago (3 children)

Not all web traffic, just the images to check. With any decent bandwidth, it shouldn't be an issue for most. It also setup in such a way as to not cause a downtime if the checker goes down.

[–] Kalcifer@sh.itjust.works 1 points 2 weeks ago

Not all web traffic, just the images to check.

Ah, yeah, my bad this was a lack of clarity on my part; I meant all image traffic.

[–] Kalcifer@sh.itjust.works 1 points 2 weeks ago (1 children)

It also setup in such a way as to not cause a downtime if the checker goes down.

Oh? Would the fallback be that it simply doesn't do a check? Or perhaps it could disable image uploads if the checker is down? Something else? Presumably, this would be configurable.

[–] db0@lemmy.dbzer0.com 2 points 2 weeks ago

It stops doing checks. Iirc you can configure it yes

[–] Kalcifer@sh.itjust.works 1 points 2 weeks ago (1 children)

With any decent bandwidth, it shouldn’t be an issue for most.

It's not only the bandwidth; I just fundamentally don't relish the idea of public traffic being directed to my local network.

[–] db0@lemmy.dbzer0.com 2 points 2 weeks ago (1 children)

You don't get public traffic redirected. It's not how it works

[–] Kalcifer@sh.itjust.works 1 points 1 week ago* (last edited 1 week ago) (1 children)

Yeah, that was poor wording on my part — what I mean to say is that there would be unvetted data flowing into my local network and being processed on a local machine. It may be overparanoia, but that feels like a privacy risk.

[–] db0@lemmy.dbzer0.com 1 points 1 week ago (1 children)

I don't see how it's a privacy risk since you're not exposing your IP or anything. Likewise the images are already uploaded to your servers, so there's no extra privacy risk for the uploader.

[–] Kalcifer@sh.itjust.works 1 points 1 week ago (1 children)

"Security risk" is probably a better term. That being said, a security risk can also infer a privacy risk.

[–] db0@lemmy.dbzer0.com 1 points 1 week ago (1 children)

Why would it be a security risk?

[–] Kalcifer@sh.itjust.works 1 points 1 week ago (1 children)

For clarity, I'm not claiming that it would, with any degree of certainty, lead to incurred damage, but the ability to upload unvetted content carries some degree of risk. For there to be no risk, fedi-safety/pictrs-safety would have to be guaranteed to be absolutely 100% free of any possible exploit, as well as the underlying OS (and maybe even the underlying hardware), which seems like an impossible claim to make, but perhaps I'm missing something important.

[–] db0@lemmy.dbzer0.com 1 points 1 week ago* (last edited 1 week ago) (1 children)

You mean an exploit payload embedded in an image, and pwning a system parsing that image through python PIL? While there's never a 100% chance of anything, you're more likely to be struck by lightning than this coming to pass and at that point you're at more security risk at using the internet altogether.

[–] Kalcifer@sh.itjust.works 1 points 5 days ago (1 children)

I will preface by saying that I am not casting doubt on your claim, I'm simply curious: What is the rationale behind why it would be so unlikely for such an exploit to occur? What rationale causes you to be so confident?

[–] db0@lemmy.dbzer0.com 2 points 4 days ago

Image processing libraries are used at the forefront of almost all web services, including lemmy and are extraordinarily robust. I really don't have the time to go at this in depth, but if you are familiar with this stuff you will know how extraordinary such an exploit would be and its existence would be causing massive chaos all over the world.

[–] db0@lemmy.dbzer0.com 3 points 2 weeks ago

https://github.com/db0/fedi-safety and the companion app https://github.com/db0/pictrs-safety which can be installed as part of your lemmy deployment in the docker-compose (or with a var in your ansible)