857
posted ago by Ferrous_Tarkus ago by Ferrous_Tarkus +857 / -0

Before I begin: let me give you my chops. I've been a Cisco Certified CCNA (at least) fellow since at least 2004. I've seen the networking and IT business evolve from the birth of the internet up until this point. I've also used Solarwinds in previous jobs... so I'm somewhat familiar with it's inner workings.

When I heard the news that Solarwinds and Orion were both compromised since April of 2020 a huge stone has sunk in my stomach and won't leave. Thus: I'm here to talk about it.

Solarwinds is used primary as a monitoring and automation system. As a monitoring system it can check internal system logs of web servers and network devices (anything that can create or write to a log file which includes network switches, routers, wi-fi access points, you name it), it can save and store those network logs, and parse those logs for things you don't want to have happen (like devices crashing or network links going down).

it can further take that output and create automated warnings to allow people like me know how bad the problem is so that we can address it in a timely fashion.

Naturally, some people have taken this a step further with automation. Is there an access point showing signs of trouble? Have Solarwinds reboot it first. Getting regular alerts about a device in timely intervals? Have a script keep records of it for a meeting next month. Constantly getting false alarms? You can suppress those.

It's that automation bit that bugs me. Depending on how integrated Solarwinds might be in a given system Solarwinds can use accounts with admin level privileges to access files and operate whole systems. If Solarwinds was indeed compromised... then ANY system that had this level of automation attached (and any "good" engineer would have done this to save time) would have been potentially remote controlled by malicious actors.

Furthermore: since this was used by every branch of the US Government along with 400+ out of the 500 Fortune 500 companies... this is basically someone potentially having the keys to the whole damn kingdom.

Comments (147)
sorted by:
You're viewing a single comment thread. View all comments, or full comment thread.
29
Razorba1 29 points ago +29 / -0

Help I’m a Solarwinds SME....MY JOB IDPS IS FUCKED

56
Ferrous_Tarkus [S] 56 points ago +56 / -0

This is what we call in the IT industry a "Resume Generating Event."

15
Rainman 15 points ago +15 / -0

Do you know if dominion used solarwinds? I saw a screen shot that showed potentially they did

17
Ferrous_Tarkus [S] 17 points ago +17 / -0

Yes.

They did.

6
emperors_apprentice 6 points ago +6 / -0

We know they used Serv-U (which in itself wasn't part of the exploit). I don't have any information if they implemented Orion or not.

2
deleted 2 points ago +2 / -0
4
jealousminarchist 4 points ago +4 / -0

Dominion had a super user root account for everyone. Solarwinds should be the smallest of their problems.

3
deleted 3 points ago +3 / -0
13
refresco 13 points ago +13 / -0

I'd start learning your competitors practices quickly and work on becoming a Solarwinds migration SME.

7
Ferrous_Tarkus [S] 7 points ago +7 / -0

Gotta be an opportunist in this industry.

3
deleted 3 points ago +3 / -0
2
squash1324 2 points ago +2 / -0

Time to write 3 letters my friend.

2
emperors_apprentice 2 points ago +2 / -0

Petition to start Sysadmin.win .. kek.

2
deleted 2 points ago +2 / -0