this post was submitted on 07 Jul 2024
1 points (100.0% liked)

Technology

59626 readers
2875 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related content.
  3. Be excellent to each another!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, to ask if your bot can be added please contact us.
  9. Check for duplicates before posting, duplicates may be removed

Approved Bots


founded 1 year ago
MODERATORS
 

I hope this won't be counted as some form of self-promotion, even though I am sharing a post from my own blog.

As a tech worker who works in a Cloud shop, I wanted to elaborate the many reasons why I find working with Clouds terrible, from multiple points of view.

I tried to organize my thoughts in a (relatively long) post, in which both technical aspects and political aspects (which are very related) are covered.

I am sure many people will have different perspectives, and this could be potentially also a nice prompt for a discussion.

top 50 comments
sorted by: hot top controversial new old
[–] Ledivin@lemmy.world 0 points 4 months ago* (last edited 4 months ago) (3 children)

I'm sorry, but this started like a recipe article and I lost all interest. I don't care about your life story, I clicked the link to read your opinions, and you spent the first several paragraphs avoiding them.

[–] loudwhisper@infosec.pub 0 points 4 months ago

Nothing to be sorry for. I didn't write for you nor for any particular individual, and it's fair if you are not interested in it. I also added a table of content at the beginning, so you can jump directly to the relevant section (Technical Side) skipping the (in my opinion needed) introduction completely, if you wish. Cheers

[–] conciselyverbose@sh.itjust.works 0 points 4 months ago

Two brief paragraphs of light nonsense on a blog post, then a quick summary of what the article will cover?

[–] Brickardo@feddit.nl 0 points 4 months ago

Tell me you don't read often without telling me you don't read often:

[–] Toes@ani.social 0 points 4 months ago (4 children)
[–] loudwhisper@infosec.pub 0 points 4 months ago (1 children)

How could I miss the opportunity to use this picture!

It definitely felt like that at times.

[–] Toes@ani.social 0 points 4 months ago

Haha, I couldn't resist. Great post though.

[–] SeeJayEmm@lemmy.procrastinati.org 0 points 4 months ago (1 children)

I was really hoping this was going to be a rant about clouds in the sky.

load more comments (1 replies)
load more comments (2 replies)
[–] iso@lemy.lol 0 points 4 months ago (1 children)

Yes, I hate cloud too. Now tell this to my company, which received about 100k dollar promotion from Azure and Google Cloud :)

[–] loudwhisper@infosec.pub 0 points 4 months ago (1 children)

What do you mean by "promotion"? A discount? Credits to get started?

[–] iso@lemy.lol 0 points 4 months ago (1 children)

Yeah credits makes more sense 👍

[–] loudwhisper@infosec.pub 0 points 4 months ago (1 children)

Oh yeah, I know that that's a thing. It's a practice not too different from the stereotypical drug dealer who gets you hooked on free drugs. In this case the idea is that if you start there, you get vendor-locked and you will have to pay that amount many times over. I understand the appeal from the company perspective, though.

[–] iso@lemy.lol 0 points 4 months ago

Yes absolutely true. For example, GKE looks very nice, but when we use one of their features, it creates the need to use other features too. That's why I warned the boss a lot. Even though they have great features, we try to use generic applications to avoid hooks.

I hope they don't take the credit back :)

[–] conciselyverbose@sh.itjust.works 0 points 4 months ago (2 children)

So the whole thing is well worth a read IMO, and addresses a lot of the issues I have with cloud as the solution for everything.

My main point here is that individuals and organizations that require all the flexibility that cloud services offer are a (tiny) minority. This means that for the majority of us, all the complexity necessary to provide this flexibility ends up being purely a complication or worse, a liability.

There are absolutely companies who need the scaling. But it's a fucking lot of overhead if you don't.

Let's repeat it one more time: complexity hides and creates security issues.

This is similar to all the LLM code stuff. If you don't actually fully understand what your code does, bad stuff happens.

This premise has the consequence that Cloud systems are a big puzzle. The pieces of the puzzle are the Cloud products. Engineers working with Cloud systems essentially need to understand the abstraction but not necessarily the underlying, ultimate working mechanism of what those abstractions do. For example, a cloud expert might know everything about the difference between NACLs and Security Groups, all the details about how to configure them, their limitations etc., but the main idea is that such expert doesn't need to know anything below that (e.g., how the traffic is filtered).

Ultimately my perspective, and I appreciate it's a very personal one, is that building and working with the Cloud makes me feel like a glorified application administrator. My job becomes researching how the Cloud solved the problem that I need to solve, and compose the solution in the way the Cloud provider imagined it should be solved, rather than solving the problem

I was going to bring up basically this point:

because vendor-lock is not something that has only to do with infrastructure. It has also to do with the skills of the engineers involved. Cloud knowledge, for the most part, is not portable. You are a wizard of IAM policies in GCP? Good job, this is completely useless if you go to Azure. Oh, you are a guru of VPCs and private endpoints? Well done, this is completely useless if you move to a different cloud.

But you covered it pretty well. Abstractions are great. Proprietary abstractions that are more focused on how they can bill you than real, useful, functional categories? Not so much.

Despite the efforts means something which is ironic: many companies which run on Cloud, at some point, will have one or more teams whose main purpose is understanding how they are spending money in the Cloud and to reduce those costs. If this sounds conflicting with the idea of reducing personnel, well, it is. The digital infrastructure of my organization is not that huge. Give or take 2000 compute instances (some very small). Something that 200 servers could easily provide. Cloud bills are more than $15 millions/year. I checked a server builder for example, and an absolute beast (something like 2x Xeon platinum processor, 200TB of NVME disks, 1TB of RAM etc.) would still stay comfortably under $250k. 100 servers this powerful will probably be a multiple of our computing power, and cost almost a third if we consider a lifetime of 3 years, which is very low. A more realistic estimation of 5 years leads to a saving of ~$50 millions over 5 years. Completely insane! This is of course if you want to buy hardware. Powerful servers rented run you for $500-1000/month. Assuming a cost of $1000/month, my company could rent more than 1000 powerful servers, and still save money compared to Cloud costs, leaving plenty for additional services such as networking, storage, premium support (remote hands) or actual engineers salary

So there's a level of rent seeking behind all the software moving to subscriptions, and them wanting to lock you in just like their service providers are doing to them. But I have to think the massive costs of cloud junk also pay a role in stuff like a calendar charging double digit annual fees for something that takes very little storage and very little computation (and you of course can't just buy software any more).

I have no words for multi-cloud. Even like a Facebook or YouTube scale site, are you really going to double (or more for some reason?) your storage costs (plus whatever intercommunication between the two), just in case the provider goes down for a couple hours (which is extremely rare, and you won't be the only site impacted, so people won't really blame you for.) Plus that architecture sounds like the shitshow to end all shitshows.

[–] loudwhisper@infosec.pub 0 points 4 months ago (1 children)

Thanks!

But I have to think the massive costs of cloud junk also pay a role in stuff like a calendar charging double digit annual fees for something that takes very little storage and very little computation (and you of course can’t just buy software any more).

Absolutely agree. I did not even think about this aspect, but I think you are absolutely spot on. Building something with huge costs is something that ultimately gets passed to the users in addition to the rent-seeking aspect.

I have no words for multi-cloud.

You and me both. I have to work with it and the reality is, there is nobody who actually understands the whole thing. The level of complexity (and fragility, I might add) of it all is astonishing. And all of this to mitigate some (honestly) low risk of downtime from the cloud provider. I have lobbied a little bit against at work, but ultimately it has become a marketing tool to sell to customers, so goodbye any hope of rational evaluation...

[–] Tja@programming.dev 0 points 4 months ago (1 children)

It's all shits and giggles until a network config takes down your cloud provider for 11 hours and you can't even look at the logs. And multicloud is quite robust if done right, more so than a single cloud, if your setup is fragile someone is not doing their job right.

[–] loudwhisper@infosec.pub 0 points 4 months ago (1 children)

Complexity brings fragility. It's not about doing the job right, is that "right" means having to deal with a level of complexity, a so high number of moving parts and configuration options, that the bar is set very high.

Also, I would argue that a large number of organizations don't actually need the resilience that they pay a very high price for.

[–] Tja@programming.dev 0 points 4 months ago (9 children)

Complexity in this case should bring redundancy, not fragility. You are adding components in parallel, not in series, thus reducing fragility.

A raid 5 is more complex than a single drive, but it's less fragile.

load more comments (9 replies)
[–] BearOfaTime@lemm.ee 0 points 4 months ago* (last edited 4 months ago)

Agreed on it all.

I think a big driver for cloud clients is bean counters - cloud is an expense, while having your own systems is capital investment.

They'd rather have the waste of leasing too much compute than have to pay taxes on systems plus the cost of staff to run it.

We won't really see this get addressed until companies have to truly own the risks they take on (see all the hacks that happen on a daily basis because CIO won't pay for the security that IT management is screaming to build). When fines for these breaches are meaningful, cloud will be less interesting.

[–] GBU_28@lemm.ee 0 points 4 months ago (1 children)

Yep. My first move is to ask "could this just live in an ec2 box? Do we really need any of aws' marketed custom options?

[–] loudwhisper@infosec.pub 0 points 4 months ago (1 children)

But then I would ask, what's the point of paying 10-20x per computing unit at that point? If you just use ec2 instance, all AWS offers you is an API to manage them, is it worth the premium? Besides, you will still need to mess with a lot of other services (VPCs, SGs, etc.) anyways.

What's the selling point in your opinion?

[–] GBU_28@lemm.ee 0 points 4 months ago (1 children)

Well I would have more questions, like why AWS at all.

But for some, cognito auth management is important, to align with other product goals.

[–] loudwhisper@infosec.pub 0 points 4 months ago

cognito auth

But then at that point you are already vendor-locked, right? At that point, running on bare ec2 instances and taking more control in your hands (vs using even more AWS-specific services) is going to help very little, when your whole user management is now tied to a specific provider.

[–] Broken@lemmy.ml 0 points 4 months ago

Thanks for sharing. Great read and points.

[–] GoOnASteamTrain@lemmy.ml 0 points 4 months ago (2 children)
[–] loudwhisper@infosec.pub 0 points 4 months ago (1 children)

This post must be fun with that one... 150+ instances in various contexts of "cloud".

[–] GoOnASteamTrain@lemmy.ml 0 points 4 months ago

Thank you for that... going back and reading again with this was very, very funny

In this case it makes sense to have a short-term quick-and-dirty Butt deployment.

[–] flambonkscious@sh.itjust.works 0 points 4 months ago

It's funny that the sheer idea or frequency of the word is distasteful enough to build this.

[–] Lettuceeatlettuce@lemmy.ml 0 points 4 months ago (3 children)

Very good read. I totally agree with your sentiment that more and more, "engineering" is becoming just gluing together and managing cloud services and features.

My job as a sys admin has become the same. It's not about actually understanding the technology at a deep level and troubleshooting problems, it's about learning specific applets and features to click on and running down daily and weekly checklists.

[–] rottingleaf@lemmy.zip 0 points 4 months ago* (last edited 4 months ago) (1 children)

“engineering” is becoming just gluing together and managing cloud services and features.

Temporarily becoming.

Just like China had some social and cultural changes since being closed and till the Opium wars.

Systems are built around people and limited by what a human can conceive and make work. We don't evolve that fast.

Also dependency on big centers has led to catastrophes in the past and will lead to those again.

It will all crash with a huge bang.

I'm confident of this, anyone who wants may call me a luddite.

[–] Lettuceeatlettuce@lemmy.ml 0 points 4 months ago

Let's hope that people will start to favor on-prem solutions and smaller independent cloud providers vs the massive trillion dollar corpo clouds that control so much now.

[–] Tja@programming.dev 0 points 4 months ago (3 children)

And that's a good thing, IMHO. As an architect I don't want to rely on some single genius knowing secret incantations or anything like that.

Boring, tried and true services, repeatedly put together and if the organization allows the time for it, with excessive documentation.

[–] conciselyverbose@sh.itjust.works 0 points 4 months ago (1 children)

No one's talking about secret incantations.

They're talking about knowing how your applications actually work, so you're not tied to the whims of a third party.

[–] Tja@programming.dev 0 points 4 months ago (1 children)

Hence or anything like that.

[–] conciselyverbose@sh.itjust.works 0 points 4 months ago (1 children)

If people don't know what your systems actually do, you're going to have huge problems at some point.

[–] Tja@programming.dev 0 points 4 months ago (4 children)

Where did I request for "not knowing what systems do"?

load more comments (4 replies)
[–] Lettuceeatlettuce@lemmy.ml 0 points 4 months ago

Straw man. I'm encountering sys admins and systems "engineers" who don't know how to spec out a server, don't understand how certificates work, don't understand basic IP addressing principles, don't understand basic networking topology.

They just know how to click a list of specific buttons in a GUI for one specific Corpo vendor.

Maybe that is fine for a Jr. Admin just starting out, but it isn't what you want for the folks in charge of building, upgrading, and maintaining your company's infrastructure.

There's nothing wrong with making interfaces simpler and easier to understand. And there's nothing wrong with building simplified abstractions on top of your systems to gain efficiency. But this should not be done at the cost of actual deep understanding and functionality.

The people you call when things go badly wrong will always be the folks that have that deep understanding and competency. It already has started hitting the developer community in the last few years. The Jr. Devs that did a 3 month boot camp where they learned nothing but how to parrot code and slap APIs together, are getting laid off and cannot find work.

The devs that went to school for Comp Sci, that have years of real world experience, and actually understand the theory and the nuts and bolts of the underlying tech, they are still largely employed and have little trouble finding work.

I think the same will happen soon in the IT world. Deep knowledge and years of dirty, greasy hands will always be desirable over a parrot that only knows how to click GUI buttons in a specific order.

[–] loudwhisper@infosec.pub 0 points 4 months ago (1 children)

Is that what you get with Cloud? Because there are still a million ways to shoot yourself in the foot. The main difference is that the single genius doesn't need to implement things him/herself, but decisions still need to be taken and fragile setups can still be built.

Imagine an ec2 instance in a satellite account performing some business critical function with an instance role, whose custom IAM policy allows to do it in another account. Clouds are not giving you good engineering, they are giving you premade building blocks, you can absolutely still make a mess with those. Even more, the complexity and the immense portfolio of features can allow very creative ways to build very low-quality systems.

I think you can have good, boring, simple systems built by engineers. With or without Cloud services.

[–] Tja@programming.dev 0 points 4 months ago

You can still make a mess, but you can't fuck up the building blocks, so it's a big improvement.

Using an ec2 instance is already a yellow flag, you have higher level services for most tasks.

[–] loudwhisper@infosec.pub 0 points 4 months ago

I feel you very much. Security work is also somewhat similar.

I think this takes a way basically the component that made it interesting, understanding what you are doing to the point that you can build stuff.

it's about learning specific applets and features to click on and running down daily and weekly checklists.

Well said.

[–] jg1i@lemmy.world 0 points 4 months ago (1 children)

I hate websites with low contrast text.

[–] loudwhisper@infosec.pub 0 points 4 months ago (1 children)

How do you get this? Anything that tries to force a light mode?

This is how the site is supposed to look like (there is no light/dark theme selection):

[–] ilmagico@lemmy.world 0 points 4 months ago* (last edited 4 months ago) (2 children)

I was reading the site on Android, and it looked dark, but after seeing this comment, I tried disabling Android system wide dark mode, and sure enough it became white like in the screenshot! For the record, I tried with both Firefox and a Chromium-based browser.

[–] loudwhisper@infosec.pub 0 points 4 months ago (3 children)

Thanks! I went and tried on my phone and indeed setting Firefox to light mode indeed causes that horrendous and unreadable result. I will need to figure out way, eventually, and provide an alternative light scheme.

load more comments (3 replies)
load more comments (1 replies)
[–] ctkatz@lemmy.ml 0 points 4 months ago (5 children)

there are too many points of failure for me to ever be comfortable using the cloud as a primary storage option.

i've always maintained this opinion when "the cloud" started being touted as being the future. and yet more corporations (including mine) are reliant on it. i mean sure, i can log in on my home computer and have some access to stuff as though i were physically at the office but that convenience ain't worth the headache if the main storage site crashes.

[–] Tja@programming.dev 0 points 4 months ago (13 children)

If the storage "crashes" it doesn't matter if it's in the cloud or on-prem.

With the cloud you get two substantial advantages:

  • the storage is built so it doesn't break so easily. I trust AWS engineers more than Mike, no matter how cool Mike is to hang out with. Additionally, if the storage breaks while Mike is on vacation we're screwed, with the cloud you get a whole team 24/7 on it.
  • you can prevent data loss with backups or multi-region setups with a few clicks/terraform lines. Try telling the PO that you need to rent datacenter space in Helsinki and Singapore for redundancy...

Of course all this costs big bucks, but technically it's superior, easier and less risky.

load more comments (13 replies)
[–] figjam@midwest.social 0 points 4 months ago

there are too many points of failure for me to ever be comfortable using the cloud as a primary storage option.

If everything that you run is local as in the same physical location and there is no requirement for external or internet access then sure. Not everyone has that luxury. Otherwise, There are the same number of points of failure in a non-cloud configuration. You just feel more comfortable with those because you have direct hands on control.

load more comments (3 replies)
load more comments
view more: next ›