Security. Open conversation among system administrators?
Good afternoon, Security is a huge concern for me at the moment. Two institutions in Ireland were targeted with Ryuk over the past few weeks. I have no doubt that there are malicious acters targeting the institution I work for right now. I was wondering if a few of you would be up for a conversation via Zoom or Teams in the coming days. I can explain what we have done and what we are continuing to do to protect ourselves. You can do the same. Some of what we are doing might overlap but potentially we all might get a few new ideas. My infrastructure might be considered legacy compared to some. I'm still using on-prem systems for the most part. But I have some cloud based services as well. All system admins / architects would be welcome. Perhaps Wednesday at 10pm gmt? I don't know what that is in other time zones. But it's probably around 6pm eastern. Regards Darragh Ó Héiligh Performing, Promoting and Sharing traditional Irish music Performance: www.darraghpipes.ie<http://www.darraghpipes.ie> Music at the Gate: www.musicatthegate.ie<http://www.musicatthegate.ie> Ceol FM: www.ceol.fm<http://www.ceol.fm> Tel: 00353(0)877670464 Email: darragh@ceol.fm
I probably wouldn't be available for a meeting due to other commitments and priorities, but I would welcome security recommendations. In my case, it's a matter of securing my personal systems at home as well as a VPS that I run via Linode. Most, if not all of what I am doing is probably obvious. I discovered yesterday that unsophisticated attackers were trying to access the server via ssh, attempting various user names, including mine. Fortunately, I've long been in the practice of disallowing password-based authentication over ssh, so they couldn't have gained unauthorized access without a key and without an exploit. Still, I wasn't comfortable, so I simply turned off ssh access over IPv4, while still allowing it over IPv6. Evidently, the attackers aren't operating against me on v6 yet. On 11/4/21 12:18 pm, Darragh Ó Héiligh wrote:
Good afternoon,
Security is a huge concern for me at the moment. Two institutions in Ireland were targeted with Ryuk over the past few weeks. I have no doubt that there are malicious acters targeting the institution I work for right now.
I was wondering if a few of you would be up for a conversation via Zoom or Teams in the coming days. I can explain what we have done and what we are continuing to do to protect ourselves. You can do the same. Some of what we are doing might overlap but potentially we all might get a few new ideas.
My infrastructure might be considered legacy compared to some. I'm still using on-prem systems for the most part. But I have some cloud based services as well. All system admins / architects would be welcome.
Perhaps Wednesday at 10pm gmt? I don't know what that is in other time zones. But it's probably around 6pm eastern.
Regards
Darragh Ó Héiligh Performing, Promoting and Sharing traditional Irish music Performance: www.darraghpipes.ie<http://www.darraghpipes.ie> Music at the Gate: www.musicatthegate.ie<http://www.musicatthegate.ie> Ceol FM: www.ceol.fm<http://www.ceol.fm>
Tel: 00353(0)877670464 Email: darragh@ceol.fm
_______________________________________________ Blind-sysadmins mailing list -- blind-sysadmins@lists.hodgsonfamily.org To unsubscribe send an email to blind-sysadmins-leave@lists.hodgsonfamily.org
That's interesting regarding IPV6. Thanks Jason. Typically, I would strongly discourage anyone from opening SSH or RDP directly to the Internet. It doesn't take much to spin up OpenVPN or a similar VPN service. This is infinitely more secure than relying on SSH. Also, 2FA for server access is a must in my opinion. -----Original Message----- From: Jason White via Blind-sysadmins <blind-sysadmins@lists.hodgsonfamily.org> Sent: Sunday 11 April 2021 17:54 To: blind-sysadmins@lists.hodgsonfamily.org Cc: Jason White <jason@jasonjgw.net> Subject: [Blind-sysadmins] Re: Security. Open conversation among system administrators? I probably wouldn't be available for a meeting due to other commitments and priorities, but I would welcome security recommendations. In my case, it's a matter of securing my personal systems at home as well as a VPS that I run via Linode. Most, if not all of what I am doing is probably obvious. I discovered yesterday that unsophisticated attackers were trying to access the server via ssh, attempting various user names, including mine. Fortunately, I've long been in the practice of disallowing password-based authentication over ssh, so they couldn't have gained unauthorized access without a key and without an exploit. Still, I wasn't comfortable, so I simply turned off ssh access over IPv4, while still allowing it over IPv6. Evidently, the attackers aren't operating against me on v6 yet. On 11/4/21 12:18 pm, Darragh Ó Héiligh wrote:
Good afternoon,
Security is a huge concern for me at the moment. Two institutions in Ireland were targeted with Ryuk over the past few weeks. I have no doubt that there are malicious acters targeting the institution I work for right now.
I was wondering if a few of you would be up for a conversation via Zoom or Teams in the coming days. I can explain what we have done and what we are continuing to do to protect ourselves. You can do the same. Some of what we are doing might overlap but potentially we all might get a few new ideas.
My infrastructure might be considered legacy compared to some. I'm still using on-prem systems for the most part. But I have some cloud based services as well. All system admins / architects would be welcome.
Perhaps Wednesday at 10pm gmt? I don't know what that is in other time zones. But it's probably around 6pm eastern.
Regards
Darragh Ó Héiligh Performing, Promoting and Sharing traditional Irish music Performance: www.darraghpipes.ie<http://www.darraghpipes.ie> Music at the Gate: www.musicatthegate.ie<http://www.musicatthegate.ie> Ceol FM: www.ceol.fm<http://www.ceol.fm>
Tel: 00353(0)877670464 Email: darragh@ceol.fm
_______________________________________________ Blind-sysadmins mailing list -- blind-sysadmins@lists.hodgsonfamily.org To unsubscribe send an email to blind-sysadmins-leave@lists.hodgsonfamily.org
_______________________________________________ Blind-sysadmins mailing list -- blind-sysadmins@lists.hodgsonfamily.org To unsubscribe send an email to blind-sysadmins-leave@lists.hodgsonfamily.org
Hi, Its only going to be a matter of time before we start seeing IPV6 attacks due to the massive rollout of IPV6. I must admit I need to do something about SSH on my servers as I currently use IP restrictions with authorised key files, as I only have one server right now didn't want to go the OpenVPN route just yet. The other thing I was looking at was port knocking in order to get SSH opened conditionally. Andrew. -----Original Message----- From: Darragh Ó Héiligh <d@digitaldarragh.com> Sent: 11 April 2021 23:04 To: Mailing list for blind system administrators <blind-sysadmins@lists.hodgsonfamily.org> Subject: [Blind-sysadmins] Re: Security. Open conversation among system administrators? That's interesting regarding IPV6. Thanks Jason. Typically, I would strongly discourage anyone from opening SSH or RDP directly to the Internet. It doesn't take much to spin up OpenVPN or a similar VPN service. This is infinitely more secure than relying on SSH. Also, 2FA for server access is a must in my opinion. -----Original Message----- From: Jason White via Blind-sysadmins <blind-sysadmins@lists.hodgsonfamily.org> Sent: Sunday 11 April 2021 17:54 To: blind-sysadmins@lists.hodgsonfamily.org Cc: Jason White <jason@jasonjgw.net> Subject: [Blind-sysadmins] Re: Security. Open conversation among system administrators? I probably wouldn't be available for a meeting due to other commitments and priorities, but I would welcome security recommendations. In my case, it's a matter of securing my personal systems at home as well as a VPS that I run via Linode. Most, if not all of what I am doing is probably obvious. I discovered yesterday that unsophisticated attackers were trying to access the server via ssh, attempting various user names, including mine. Fortunately, I've long been in the practice of disallowing password-based authentication over ssh, so they couldn't have gained unauthorized access without a key and without an exploit. Still, I wasn't comfortable, so I simply turned off ssh access over IPv4, while still allowing it over IPv6. Evidently, the attackers aren't operating against me on v6 yet. On 11/4/21 12:18 pm, Darragh Ó Héiligh wrote:
Good afternoon,
Security is a huge concern for me at the moment. Two institutions in Ireland were targeted with Ryuk over the past few weeks. I have no doubt that there are malicious acters targeting the institution I work for right now.
I was wondering if a few of you would be up for a conversation via Zoom or Teams in the coming days. I can explain what we have done and what we are continuing to do to protect ourselves. You can do the same. Some of what we are doing might overlap but potentially we all might get a few new ideas.
My infrastructure might be considered legacy compared to some. I'm still using on-prem systems for the most part. But I have some cloud based services as well. All system admins / architects would be welcome.
Perhaps Wednesday at 10pm gmt? I don't know what that is in other time zones. But it's probably around 6pm eastern.
Regards
Darragh Ó Héiligh Performing, Promoting and Sharing traditional Irish music Performance: www.darraghpipes.ie<http://www.darraghpipes.ie> Music at the Gate: www.musicatthegate.ie<http://www.musicatthegate.ie> Ceol FM: www.ceol.fm<http://www.ceol.fm>
Tel: 00353(0)877670464 Email: darragh@ceol.fm
_______________________________________________ Blind-sysadmins mailing list -- blind-sysadmins@lists.hodgsonfamily.org To unsubscribe send an email to blind-sysadmins-leave@lists.hodgsonfamily.org
_______________________________________________ Blind-sysadmins mailing list -- blind-sysadmins@lists.hodgsonfamily.org To unsubscribe send an email to blind-sysadmins-leave@lists.hodgsonfamily.org _______________________________________________ Blind-sysadmins mailing list -- blind-sysadmins@lists.hodgsonfamily.org To unsubscribe send an email to blind-sysadmins-leave@lists.hodgsonfamily.org
It seems to me either your ssh server is secure against password guessing programs or it is not. If it's possible for someone to get into your system by pointing a password guessing program at it, you ought to fix that. If it is not possible, then there was no reason to turn off IPV4. I think people worry about this stuff too much. Before you freak out about what I just said, understand that I am not saying hacking isn't a huge problem. I'm saying worrying doesn't help. If you disabled password logins on your ssh server, why are you worried about someone trying to guess a password? I pretty regularly have conversations with other sys admins who don't understand how I can not worry about this stuff. I don't even bother checking if hackers are pointing password guessing programs at my server, I know they are. I'd bet on it. But I've disabled password login so I don't worry about it. What if it turns out there's a vulnerability in OpenSSH to allow password logins even if you disable it? Well, there isn't. Suppose there is. Okay, well, then most likely, hackers would be breaking into Citibank and DARPA, not my systems, and I'd find out about it when the New York Times blasted it on the front page. What if the bug was so well hidden that hackers had even gotten around to your systems? Then I'd just kick them off. I've got backups. The scenario above isn't all that unrealistic. Something close occured in 2014 with the heartbleed bug. I just remained calm, made sure my systems hadn't been hacked, installed the fix, and moved on. I think the way to get to the point where you can be confident in your systems is by getting to the point where you're not worried hackers know something you don't. And the way to do that is to get connected with the ethical hacker community. Hackers are smart but so are you. Hackers know stuff but so do you. I found the group I'm in by taking the ethical hacking course at the local community college. The class itself was useful but the contacts I made were invaluable. On 4/11/21 11:53 AM, blind-sysadmins@lists.hodgsonfamily.org wrote:
I probably wouldn't be available for a meeting due to other commitments and priorities, but I would welcome security recommendations. In my case, it's a matter of securing my personal systems at home as well as a VPS that I run via Linode.
Most, if not all of what I am doing is probably obvious. I discovered yesterday that unsophisticated attackers were trying to access the server via ssh, attempting various user names, including mine. Fortunately, I've long been in the practice of disallowing password-based authentication over ssh, so they couldn't have gained unauthorized access without a key and without an exploit. Still, I wasn't comfortable, so I simply turned off ssh access over IPv4, while still allowing it over IPv6. Evidently, the attackers aren't operating against me on v6 yet.
On 11/4/21 12:18 pm, Darragh Ó Héiligh wrote:
Good afternoon,
Security is a huge concern for me at the moment. Two institutions in Ireland were targeted with Ryuk over the past few weeks. I have no doubt that there are malicious acters targeting the institution I work for right now.
I was wondering if a few of you would be up for a conversation via Zoom or Teams in the coming days. I can explain what we have done and what we are continuing to do to protect ourselves. You can do the same. Some of what we are doing might overlap but potentially we all might get a few new ideas.
My infrastructure might be considered legacy compared to some. I'm still using on-prem systems for the most part. But I have some cloud based services as well. All system admins / architects would be welcome.
Perhaps Wednesday at 10pm gmt? I don't know what that is in other time zones. But it's probably around 6pm eastern.
Regards
Darragh Ó Héiligh Performing, Promoting and Sharing traditional Irish music Performance: www.darraghpipes.ie<http://www.darraghpipes.ie> Music at the Gate: www.musicatthegate.ie<http://www.musicatthegate.ie> Ceol FM: www.ceol.fm<http://www.ceol.fm>
Tel: 00353(0)877670464 Email: darragh@ceol.fm
_______________________________________________ Blind-sysadmins mailing list -- blind-sysadmins@lists.hodgsonfamily.org To unsubscribe send an email to blind-sysadmins-leave@lists.hodgsonfamily.org
_______________________________________________ Blind-sysadmins mailing list -- blind-sysadmins@lists.hodgsonfamily.org To unsubscribe send an email to blind-sysadmins-leave@lists.hodgsonfamily.org
Hi, Only reason why I would be interested in turning this off is to not pollute the logs and miss other important information and to also reduce the traffic coming to the server which may be important if I was constrained on bandwidth (due to cost) or CPU cycles. I tend to agree with you in the main which is why I haven't done anything about it on my end. Andrew. -----Original Message----- From: John G Heim via Blind-sysadmins <blind-sysadmins@lists.hodgsonfamily.org> Sent: 12 April 2021 16:39 To: Mailing list for blind system administrators <blind-sysadmins@lists.hodgsonfamily.org> Cc: John G Heim <jheim@math.wisc.edu> Subject: [Blind-sysadmins] Re: Security. Open conversation among system administrators? It seems to me either your ssh server is secure against password guessing programs or it is not. If it's possible for someone to get into your system by pointing a password guessing program at it, you ought to fix that. If it is not possible, then there was no reason to turn off IPV4. I think people worry about this stuff too much. Before you freak out about what I just said, understand that I am not saying hacking isn't a huge problem. I'm saying worrying doesn't help. If you disabled password logins on your ssh server, why are you worried about someone trying to guess a password? I pretty regularly have conversations with other sys admins who don't understand how I can not worry about this stuff. I don't even bother checking if hackers are pointing password guessing programs at my server, I know they are. I'd bet on it. But I've disabled password login so I don't worry about it. What if it turns out there's a vulnerability in OpenSSH to allow password logins even if you disable it? Well, there isn't. Suppose there is. Okay, well, then most likely, hackers would be breaking into Citibank and DARPA, not my systems, and I'd find out about it when the New York Times blasted it on the front page. What if the bug was so well hidden that hackers had even gotten around to your systems? Then I'd just kick them off. I've got backups. The scenario above isn't all that unrealistic. Something close occured in 2014 with the heartbleed bug. I just remained calm, made sure my systems hadn't been hacked, installed the fix, and moved on. I think the way to get to the point where you can be confident in your systems is by getting to the point where you're not worried hackers know something you don't. And the way to do that is to get connected with the ethical hacker community. Hackers are smart but so are you. Hackers know stuff but so do you. I found the group I'm in by taking the ethical hacking course at the local community college. The class itself was useful but the contacts I made were invaluable. On 4/11/21 11:53 AM, blind-sysadmins@lists.hodgsonfamily.org wrote:
I probably wouldn't be available for a meeting due to other commitments and priorities, but I would welcome security recommendations. In my case, it's a matter of securing my personal systems at home as well as a VPS that I run via Linode.
Most, if not all of what I am doing is probably obvious. I discovered yesterday that unsophisticated attackers were trying to access the server via ssh, attempting various user names, including mine. Fortunately, I've long been in the practice of disallowing password-based authentication over ssh, so they couldn't have gained unauthorized access without a key and without an exploit. Still, I wasn't comfortable, so I simply turned off ssh access over IPv4, while still allowing it over IPv6. Evidently, the attackers aren't operating against me on v6 yet.
On 11/4/21 12:18 pm, Darragh Ó Héiligh wrote:
Good afternoon,
Security is a huge concern for me at the moment. Two institutions in Ireland were targeted with Ryuk over the past few weeks. I have no doubt that there are malicious acters targeting the institution I work for right now.
I was wondering if a few of you would be up for a conversation via Zoom or Teams in the coming days. I can explain what we have done and what we are continuing to do to protect ourselves. You can do the same. Some of what we are doing might overlap but potentially we all might get a few new ideas.
My infrastructure might be considered legacy compared to some. I'm still using on-prem systems for the most part. But I have some cloud based services as well. All system admins / architects would be welcome.
Perhaps Wednesday at 10pm gmt? I don't know what that is in other time zones. But it's probably around 6pm eastern.
Regards
Darragh Ó Héiligh Performing, Promoting and Sharing traditional Irish music Performance: www.darraghpipes.ie<http://www.darraghpipes.ie> Music at the Gate: www.musicatthegate.ie<http://www.musicatthegate.ie> Ceol FM: www.ceol.fm<http://www.ceol.fm>
Tel: 00353(0)877670464 Email: darragh@ceol.fm
_______________________________________________ Blind-sysadmins mailing list -- blind-sysadmins@lists.hodgsonfamily.org To unsubscribe send an email to blind-sysadmins-leave@lists.hodgsonfamily.org
_______________________________________________ Blind-sysadmins mailing list -- blind-sysadmins@lists.hodgsonfamily.org To unsubscribe send an email to blind-sysadmins-leave@lists.hodgsonfamily.org
Blind-sysadmins mailing list -- blind-sysadmins@lists.hodgsonfamily.org To unsubscribe send an email to blind-sysadmins-leave@lists.hodgsonfamily.org
Some of this guidance may be helpful: https://www.cisa.gov/sites/default/files/publications/CISA_MS-ISAC_Ransomwar... Chris On Sun, Apr 11, 2021 at 04:18:59PM +0000, Darragh ? H?iligh wrote:
Good afternoon,
Security is a huge concern for me at the moment. Two institutions in Ireland were targeted with Ryuk over the past few weeks. I have no doubt that there are malicious acters targeting the institution I work for right now.
I was wondering if a few of you would be up for a conversation via Zoom or Teams in the coming days. I can explain what we have done and what we are continuing to do to protect ourselves. You can do the same. Some of what we are doing might overlap but potentially we all might get a few new ideas.
My infrastructure might be considered legacy compared to some. I'm still using on-prem systems for the most part. But I have some cloud based services as well. All system admins / architects would be welcome.
Perhaps Wednesday at 10pm gmt? I don't know what that is in other time zones. But it's probably around 6pm eastern.
Regards
Darragh ? H?iligh Performing, Promoting and Sharing traditional Irish music Performance: www.darraghpipes.ie<http://www.darraghpipes.ie> Music at the Gate: www.musicatthegate.ie<http://www.musicatthegate.ie> Ceol FM: www.ceol.fm<http://www.ceol.fm>
Tel: 00353(0)877670464 Email: darragh@ceol.fm
_______________________________________________ Blind-sysadmins mailing list -- blind-sysadmins@lists.hodgsonfamily.org To unsubscribe send an email to blind-sysadmins-leave@lists.hodgsonfamily.org
Hi, Interesting idea. A bit late in the evening for me though could manage it if others were interested. The company I work for are very concerned about possible ransomware attacks. In the main they are trying to mitigate against this by disabling admin privs everywhere and only allowing them for the time they are required after an approval process, this is across the board and has had several interesting ramifications. Andrew. -----Original Message----- From: Darragh Ó Héiligh <d@digitaldarragh.com> Sent: 11 April 2021 17:19 To: Mailing list for blind system administrators <blind-sysadmins@lists.hodgsonfamily.org> Subject: [Blind-sysadmins] Security. Open conversation among system administrators? Good afternoon, Security is a huge concern for me at the moment. Two institutions in Ireland were targeted with Ryuk over the past few weeks. I have no doubt that there are malicious acters targeting the institution I work for right now. I was wondering if a few of you would be up for a conversation via Zoom or Teams in the coming days. I can explain what we have done and what we are continuing to do to protect ourselves. You can do the same. Some of what we are doing might overlap but potentially we all might get a few new ideas. My infrastructure might be considered legacy compared to some. I'm still using on-prem systems for the most part. But I have some cloud based services as well. All system admins / architects would be welcome. Perhaps Wednesday at 10pm gmt? I don't know what that is in other time zones. But it's probably around 6pm eastern. Regards Darragh Ó Héiligh Performing, Promoting and Sharing traditional Irish music Performance: www.darraghpipes.ie<http://www.darraghpipes.ie> Music at the Gate: www.musicatthegate.ie<http://www.musicatthegate.ie> Ceol FM: www.ceol.fm<http://www.ceol.fm> Tel: 00353(0)877670464 Email: darragh@ceol.fm _______________________________________________ Blind-sysadmins mailing list -- blind-sysadmins@lists.hodgsonfamily.org To unsubscribe send an email to blind-sysadmins-leave@lists.hodgsonfamily.org
Hey Andrew, What are your people using to provide just in time access?? I'm going to look up products tomorrow. I was thinking of just writing something myself but I'm conscious of not spending hours on coding when there's so many other things to do. -----Original Message----- From: Andrew Hodgson <andrew@hodgson.io> Sent: Sunday 11 April 2021 22:35 To: Mailing list for blind system administrators <blind-sysadmins@lists.hodgsonfamily.org> Subject: [Blind-sysadmins] Re: Security. Open conversation among system administrators? Hi, Interesting idea. A bit late in the evening for me though could manage it if others were interested. The company I work for are very concerned about possible ransomware attacks. In the main they are trying to mitigate against this by disabling admin privs everywhere and only allowing them for the time they are required after an approval process, this is across the board and has had several interesting ramifications. Andrew. -----Original Message----- From: Darragh Ó Héiligh <d@digitaldarragh.com> Sent: 11 April 2021 17:19 To: Mailing list for blind system administrators <blind-sysadmins@lists.hodgsonfamily.org> Subject: [Blind-sysadmins] Security. Open conversation among system administrators? Good afternoon, Security is a huge concern for me at the moment. Two institutions in Ireland were targeted with Ryuk over the past few weeks. I have no doubt that there are malicious acters targeting the institution I work for right now. I was wondering if a few of you would be up for a conversation via Zoom or Teams in the coming days. I can explain what we have done and what we are continuing to do to protect ourselves. You can do the same. Some of what we are doing might overlap but potentially we all might get a few new ideas. My infrastructure might be considered legacy compared to some. I'm still using on-prem systems for the most part. But I have some cloud based services as well. All system admins / architects would be welcome. Perhaps Wednesday at 10pm gmt? I don't know what that is in other time zones. But it's probably around 6pm eastern. Regards Darragh Ó Héiligh Performing, Promoting and Sharing traditional Irish music Performance: www.darraghpipes.ie<http://www.darraghpipes.ie> Music at the Gate: www.musicatthegate.ie<http://www.musicatthegate.ie> Ceol FM: www.ceol.fm<http://www.ceol.fm> Tel: 00353(0)877670464 Email: darragh@ceol.fm _______________________________________________ Blind-sysadmins mailing list -- blind-sysadmins@lists.hodgsonfamily.org To unsubscribe send an email to blind-sysadmins-leave@lists.hodgsonfamily.org _______________________________________________ Blind-sysadmins mailing list -- blind-sysadmins@lists.hodgsonfamily.org To unsubscribe send an email to blind-sysadmins-leave@lists.hodgsonfamily.org
Hi, So from my side I have been mainly working on the Azure parts so that has been using Priviledged Access Management as part of Azure Active Directory to get a specific set of members allowable access to a group for a certain time. https://docs.microsoft.com/en-us/azure/active-directory/privileged-identity-... I have got this working for access to Kubernetes clusters and also to allow access to a specific Azure AAD role required to create specific types of SPNs. From the Azure VM side I have been experimenting with JIT access for NSGs: https://docs.microsoft.com/en-us/azure/security-center/security-center-just-... This is where I can grant a specific IP belonging to an administrator direct access into the VM for management purposes via SSH or RDP. If I'm honest I think that VPN access in is better than allowing IP address via NSG especially as we're looking at this method to allow access to a bastion host which causes me problems myself. One advantage with this type of access is I have been experimenting with using Azure Devops public hosted agents with this so we don't have to host our own agents. I realise that is a bit out of scope for what you want. I have also been advocating that any admin access to the AAD tenant for SPN creation or priviledged roles can be scripted and put through a pipeline. The pipeline runs from an account with only privs to do the necessary functions. I've run into a bit of trouble with this one as the initial design I chose for this meant we granted very specific API permissions to perform the necessary actions but I kept running into issues with more privs required for a specific purpose or for example if providing an account to a team to perform a specific function like creation of AAD apps depending on which tool they used for the job they would end up with more perms required. In terms of end user devices there is a team playing with a piece of software called Osirium Priviledged Endpoint Management: https://www.osirium.com/products/privileged-endpoint-management This software allows a user to step up to an admin group for a set amount of time based on access via its console. I find this software fairly inaccessible if I'm honest. A group of us have decided to disjoin ourselves from the on-prem domain and use AAD join with InTune managing the software and other settings on the laptops aka group policy. Its been successful for me so far but my workflow doesn't require me to do anything with Windows management or Active Directory on-prem work. May be an option for developer workloads though. Hope that has given you some ideas. Andrew. -----Original Message----- From: Darragh Ó Héiligh <d@digitaldarragh.com> Sent: 11 April 2021 23:05 To: Mailing list for blind system administrators <blind-sysadmins@lists.hodgsonfamily.org> Subject: [Blind-sysadmins] Re: Security. Open conversation among system administrators? Hey Andrew, What are your people using to provide just in time access?? I'm going to look up products tomorrow. I was thinking of just writing something myself but I'm conscious of not spending hours on coding when there's so many other things to do. -----Original Message----- From: Andrew Hodgson <andrew@hodgson.io> Sent: Sunday 11 April 2021 22:35 To: Mailing list for blind system administrators <blind-sysadmins@lists.hodgsonfamily.org> Subject: [Blind-sysadmins] Re: Security. Open conversation among system administrators? Hi, Interesting idea. A bit late in the evening for me though could manage it if others were interested. The company I work for are very concerned about possible ransomware attacks. In the main they are trying to mitigate against this by disabling admin privs everywhere and only allowing them for the time they are required after an approval process, this is across the board and has had several interesting ramifications. Andrew. -----Original Message----- From: Darragh Ó Héiligh <d@digitaldarragh.com> Sent: 11 April 2021 17:19 To: Mailing list for blind system administrators <blind-sysadmins@lists.hodgsonfamily.org> Subject: [Blind-sysadmins] Security. Open conversation among system administrators? Good afternoon, Security is a huge concern for me at the moment. Two institutions in Ireland were targeted with Ryuk over the past few weeks. I have no doubt that there are malicious acters targeting the institution I work for right now. I was wondering if a few of you would be up for a conversation via Zoom or Teams in the coming days. I can explain what we have done and what we are continuing to do to protect ourselves. You can do the same. Some of what we are doing might overlap but potentially we all might get a few new ideas. My infrastructure might be considered legacy compared to some. I'm still using on-prem systems for the most part. But I have some cloud based services as well. All system admins / architects would be welcome. Perhaps Wednesday at 10pm gmt? I don't know what that is in other time zones. But it's probably around 6pm eastern. Regards Darragh Ó Héiligh Performing, Promoting and Sharing traditional Irish music Performance: www.darraghpipes.ie<http://www.darraghpipes.ie> Music at the Gate: www.musicatthegate.ie<http://www.musicatthegate.ie> Ceol FM: www.ceol.fm<http://www.ceol.fm> Tel: 00353(0)877670464 Email: darragh@ceol.fm _______________________________________________ Blind-sysadmins mailing list -- blind-sysadmins@lists.hodgsonfamily.org To unsubscribe send an email to blind-sysadmins-leave@lists.hodgsonfamily.org _______________________________________________ Blind-sysadmins mailing list -- blind-sysadmins@lists.hodgsonfamily.org To unsubscribe send an email to blind-sysadmins-leave@lists.hodgsonfamily.org _______________________________________________ Blind-sysadmins mailing list -- blind-sysadmins@lists.hodgsonfamily.org To unsubscribe send an email to blind-sysadmins-leave@lists.hodgsonfamily.org
Yeah. This is really interesting. Thanks. It would be interesting to look at using an Azure AD based solution for temporary elevation. But that would take time to propagate to the on-prem domain. My sys admins aren't really delighted with these changes so if I told them they would have to wait for up to 15 minutes before getting the access they need, they might revolt. -----Original Message----- From: Andrew Hodgson <andrew@hodgson.io> Sent: Monday 12 April 2021 01:11 To: Mailing list for blind system administrators <blind-sysadmins@lists.hodgsonfamily.org> Subject: [Blind-sysadmins] Re: Security. Open conversation among system administrators? Hi, So from my side I have been mainly working on the Azure parts so that has been using Priviledged Access Management as part of Azure Active Directory to get a specific set of members allowable access to a group for a certain time. https://docs.microsoft.com/en-us/azure/active-directory/privileged-identity-... I have got this working for access to Kubernetes clusters and also to allow access to a specific Azure AAD role required to create specific types of SPNs. From the Azure VM side I have been experimenting with JIT access for NSGs: https://docs.microsoft.com/en-us/azure/security-center/security-center-just-... This is where I can grant a specific IP belonging to an administrator direct access into the VM for management purposes via SSH or RDP. If I'm honest I think that VPN access in is better than allowing IP address via NSG especially as we're looking at this method to allow access to a bastion host which causes me problems myself. One advantage with this type of access is I have been experimenting with using Azure Devops public hosted agents with this so we don't have to host our own agents. I realise that is a bit out of scope for what you want. I have also been advocating that any admin access to the AAD tenant for SPN creation or priviledged roles can be scripted and put through a pipeline. The pipeline runs from an account with only privs to do the necessary functions. I've run into a bit of trouble with this one as the initial design I chose for this meant we granted very specific API permissions to perform the necessary actions but I kept running into issues with more privs required for a specific purpose or for example if providing an account to a team to perform a specific function like creation of AAD apps depending on which tool they used for the job they would end up with more perms required. In terms of end user devices there is a team playing with a piece of software called Osirium Priviledged Endpoint Management: https://www.osirium.com/products/privileged-endpoint-management This software allows a user to step up to an admin group for a set amount of time based on access via its console. I find this software fairly inaccessible if I'm honest. A group of us have decided to disjoin ourselves from the on-prem domain and use AAD join with InTune managing the software and other settings on the laptops aka group policy. Its been successful for me so far but my workflow doesn't require me to do anything with Windows management or Active Directory on-prem work. May be an option for developer workloads though. Hope that has given you some ideas. Andrew. -----Original Message----- From: Darragh Ó Héiligh <d@digitaldarragh.com> Sent: 11 April 2021 23:05 To: Mailing list for blind system administrators <blind-sysadmins@lists.hodgsonfamily.org> Subject: [Blind-sysadmins] Re: Security. Open conversation among system administrators? Hey Andrew, What are your people using to provide just in time access?? I'm going to look up products tomorrow. I was thinking of just writing something myself but I'm conscious of not spending hours on coding when there's so many other things to do. -----Original Message----- From: Andrew Hodgson <andrew@hodgson.io> Sent: Sunday 11 April 2021 22:35 To: Mailing list for blind system administrators <blind-sysadmins@lists.hodgsonfamily.org> Subject: [Blind-sysadmins] Re: Security. Open conversation among system administrators? Hi, Interesting idea. A bit late in the evening for me though could manage it if others were interested. The company I work for are very concerned about possible ransomware attacks. In the main they are trying to mitigate against this by disabling admin privs everywhere and only allowing them for the time they are required after an approval process, this is across the board and has had several interesting ramifications. Andrew. -----Original Message----- From: Darragh Ó Héiligh <d@digitaldarragh.com> Sent: 11 April 2021 17:19 To: Mailing list for blind system administrators <blind-sysadmins@lists.hodgsonfamily.org> Subject: [Blind-sysadmins] Security. Open conversation among system administrators? Good afternoon, Security is a huge concern for me at the moment. Two institutions in Ireland were targeted with Ryuk over the past few weeks. I have no doubt that there are malicious acters targeting the institution I work for right now. I was wondering if a few of you would be up for a conversation via Zoom or Teams in the coming days. I can explain what we have done and what we are continuing to do to protect ourselves. You can do the same. Some of what we are doing might overlap but potentially we all might get a few new ideas. My infrastructure might be considered legacy compared to some. I'm still using on-prem systems for the most part. But I have some cloud based services as well. All system admins / architects would be welcome. Perhaps Wednesday at 10pm gmt? I don't know what that is in other time zones. But it's probably around 6pm eastern. Regards Darragh Ó Héiligh Performing, Promoting and Sharing traditional Irish music Performance: www.darraghpipes.ie<http://www.darraghpipes.ie> Music at the Gate: www.musicatthegate.ie<http://www.musicatthegate.ie> Ceol FM: www.ceol.fm<http://www.ceol.fm> Tel: 00353(0)877670464 Email: darragh@ceol.fm _______________________________________________ Blind-sysadmins mailing list -- blind-sysadmins@lists.hodgsonfamily.org To unsubscribe send an email to blind-sysadmins-leave@lists.hodgsonfamily.org _______________________________________________ Blind-sysadmins mailing list -- blind-sysadmins@lists.hodgsonfamily.org To unsubscribe send an email to blind-sysadmins-leave@lists.hodgsonfamily.org _______________________________________________ Blind-sysadmins mailing list -- blind-sysadmins@lists.hodgsonfamily.org To unsubscribe send an email to blind-sysadmins-leave@lists.hodgsonfamily.org _______________________________________________ Blind-sysadmins mailing list -- blind-sysadmins@lists.hodgsonfamily.org To unsubscribe send an email to blind-sysadmins-leave@lists.hodgsonfamily.org
Hi, Yeah and I don't think that is possible either with the method I proposed. I have no idea what is being done on the on-prem domain controllers to be honest, that isn't my department right now. Andrew. -----Original Message----- From: Darragh Ó Héiligh <d@digitaldarragh.com> Sent: 12 April 2021 22:22 To: Mailing list for blind system administrators <blind-sysadmins@lists.hodgsonfamily.org> Subject: [Blind-sysadmins] Re: Security. Open conversation among system administrators? Yeah. This is really interesting. Thanks. It would be interesting to look at using an Azure AD based solution for temporary elevation. But that would take time to propagate to the on-prem domain. My sys admins aren't really delighted with these changes so if I told them they would have to wait for up to 15 minutes before getting the access they need, they might revolt. -----Original Message----- From: Andrew Hodgson <andrew@hodgson.io> Sent: Monday 12 April 2021 01:11 To: Mailing list for blind system administrators <blind-sysadmins@lists.hodgsonfamily.org> Subject: [Blind-sysadmins] Re: Security. Open conversation among system administrators? Hi, So from my side I have been mainly working on the Azure parts so that has been using Priviledged Access Management as part of Azure Active Directory to get a specific set of members allowable access to a group for a certain time. https://docs.microsoft.com/en-us/azure/active-directory/privileged-identity-... I have got this working for access to Kubernetes clusters and also to allow access to a specific Azure AAD role required to create specific types of SPNs. From the Azure VM side I have been experimenting with JIT access for NSGs: https://docs.microsoft.com/en-us/azure/security-center/security-center-just-... This is where I can grant a specific IP belonging to an administrator direct access into the VM for management purposes via SSH or RDP. If I'm honest I think that VPN access in is better than allowing IP address via NSG especially as we're looking at this method to allow access to a bastion host which causes me problems myself. One advantage with this type of access is I have been experimenting with using Azure Devops public hosted agents with this so we don't have to host our own agents. I realise that is a bit out of scope for what you want. I have also been advocating that any admin access to the AAD tenant for SPN creation or priviledged roles can be scripted and put through a pipeline. The pipeline runs from an account with only privs to do the necessary functions. I've run into a bit of trouble with this one as the initial design I chose for this meant we granted very specific API permissions to perform the necessary actions but I kept running into issues with more privs required for a specific purpose or for example if providing an account to a team to perform a specific function like creation of AAD apps depending on which tool they used for the job they would end up with more perms required. In terms of end user devices there is a team playing with a piece of software called Osirium Priviledged Endpoint Management: https://www.osirium.com/products/privileged-endpoint-management This software allows a user to step up to an admin group for a set amount of time based on access via its console. I find this software fairly inaccessible if I'm honest. A group of us have decided to disjoin ourselves from the on-prem domain and use AAD join with InTune managing the software and other settings on the laptops aka group policy. Its been successful for me so far but my workflow doesn't require me to do anything with Windows management or Active Directory on-prem work. May be an option for developer workloads though. Hope that has given you some ideas. Andrew. -----Original Message----- From: Darragh Ó Héiligh <d@digitaldarragh.com> Sent: 11 April 2021 23:05 To: Mailing list for blind system administrators <blind-sysadmins@lists.hodgsonfamily.org> Subject: [Blind-sysadmins] Re: Security. Open conversation among system administrators? Hey Andrew, What are your people using to provide just in time access?? I'm going to look up products tomorrow. I was thinking of just writing something myself but I'm conscious of not spending hours on coding when there's so many other things to do. -----Original Message----- From: Andrew Hodgson <andrew@hodgson.io> Sent: Sunday 11 April 2021 22:35 To: Mailing list for blind system administrators <blind-sysadmins@lists.hodgsonfamily.org> Subject: [Blind-sysadmins] Re: Security. Open conversation among system administrators? Hi, Interesting idea. A bit late in the evening for me though could manage it if others were interested. The company I work for are very concerned about possible ransomware attacks. In the main they are trying to mitigate against this by disabling admin privs everywhere and only allowing them for the time they are required after an approval process, this is across the board and has had several interesting ramifications. Andrew. -----Original Message----- From: Darragh Ó Héiligh <d@digitaldarragh.com> Sent: 11 April 2021 17:19 To: Mailing list for blind system administrators <blind-sysadmins@lists.hodgsonfamily.org> Subject: [Blind-sysadmins] Security. Open conversation among system administrators? Good afternoon, Security is a huge concern for me at the moment. Two institutions in Ireland were targeted with Ryuk over the past few weeks. I have no doubt that there are malicious acters targeting the institution I work for right now. I was wondering if a few of you would be up for a conversation via Zoom or Teams in the coming days. I can explain what we have done and what we are continuing to do to protect ourselves. You can do the same. Some of what we are doing might overlap but potentially we all might get a few new ideas. My infrastructure might be considered legacy compared to some. I'm still using on-prem systems for the most part. But I have some cloud based services as well. All system admins / architects would be welcome. Perhaps Wednesday at 10pm gmt? I don't know what that is in other time zones. But it's probably around 6pm eastern. Regards Darragh Ó Héiligh Performing, Promoting and Sharing traditional Irish music Performance: www.darraghpipes.ie<http://www.darraghpipes.ie> Music at the Gate: www.musicatthegate.ie<http://www.musicatthegate.ie> Ceol FM: www.ceol.fm<http://www.ceol.fm> Tel: 00353(0)877670464 Email: darragh@ceol.fm _______________________________________________ Blind-sysadmins mailing list -- blind-sysadmins@lists.hodgsonfamily.org To unsubscribe send an email to blind-sysadmins-leave@lists.hodgsonfamily.org _______________________________________________ Blind-sysadmins mailing list -- blind-sysadmins@lists.hodgsonfamily.org To unsubscribe send an email to blind-sysadmins-leave@lists.hodgsonfamily.org _______________________________________________ Blind-sysadmins mailing list -- blind-sysadmins@lists.hodgsonfamily.org To unsubscribe send an email to blind-sysadmins-leave@lists.hodgsonfamily.org _______________________________________________ Blind-sysadmins mailing list -- blind-sysadmins@lists.hodgsonfamily.org To unsubscribe send an email to blind-sysadmins-leave@lists.hodgsonfamily.org _______________________________________________ Blind-sysadmins mailing list -- blind-sysadmins@lists.hodgsonfamily.org To unsubscribe send an email to blind-sysadmins-leave@lists.hodgsonfamily.org
Hi Darragh, Something I would definitely be interested in, although as Andrew as said may be a little late in the day for me. Security wise we're looking towards focusing on admin privileges / inactive accounts within on prem AD to give any possible ransomware attacks as little to work with as possible. We're also working towards getting a cyber essentials certificate. We have also floated around the idea of 2FA being used for any admin tasks / domain admin logins, however this hasn't gone any further yet. Azure AD wise 2FA, conditional access and information protection are enabled and used across the board. AV wise we use Symantec endpoint protection on prem, however as a security team we are pushing towards their cloud offering to help us protect more devices including mobiles / tablets. For managing vulnerabilities we have a Nessus licence that runs a daily scan against our internal server subnets and then pushes those results into a bespoke dashboard, and we also use outpost 24 for external PCIDSS related scanning which we're also working on incorporating into the same dashboard. We recently saw evidence of a password spraying attack against our domain but we're still trying to investigate the origins. Luckily no access was granted as the targeted accounts locked themselves almost immediately. We work closely with the NCSC also and they have quite a lot of good tools e.g. early warning service, logging made easy, etc so it may be worth checking those out. Thanks, Kieran. Kieran Little IS Support Technician (Solutions Design Assurance) Information Services Northumberland County Council County Hall Morpeth NE61 2EF tel: 01670 623699 Mobile: 07966325130 Chat in teams Email: kieran.little@northumberland.gov.uk Website: www.northumberland.gov.uk -----Original Message----- From: Darragh Ó Héiligh <d@digitaldarragh.com> Sent: 11 April 2021 17:19 To: Mailing list for blind system administrators <blind-sysadmins@lists.hodgsonfamily.org> Subject: [Blind-sysadmins] Security. Open conversation among system administrators? Good afternoon, Security is a huge concern for me at the moment. Two institutions in Ireland were targeted with Ryuk over the past few weeks. I have no doubt that there are malicious acters targeting the institution I work for right now. I was wondering if a few of you would be up for a conversation via Zoom or Teams in the coming days. I can explain what we have done and what we are continuing to do to protect ourselves. You can do the same. Some of what we are doing might overlap but potentially we all might get a few new ideas. My infrastructure might be considered legacy compared to some. I'm still using on-prem systems for the most part. But I have some cloud based services as well. All system admins / architects would be welcome. Perhaps Wednesday at 10pm gmt? I don't know what that is in other time zones. But it's probably around 6pm eastern. Regards Darragh Ó Héiligh Performing, Promoting and Sharing traditional Irish music Performance: https://gbr01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.darraghpipes.ie%2F&data=04%7C01%7Ckieran.little%40northumberland.gov.uk%7C2d3d98b5a4d24a8c728108d8fd059a9e%7Cbb13a9de829042f0a980dc3bdfe70f40%7C0%7C0%7C637537547794353744%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=ThSNmb196ym4zG1rlajAZFEJKjfifWmte3jcDRedgic%3D&reserved=0<https://gbr01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.darraghpipes.ie%2F&data=04%7C01%7Ckieran.little%40northumberland.gov.uk%7C2d3d98b5a4d24a8c728108d8fd059a9e%7Cbb13a9de829042f0a980dc3bdfe70f40%7C0%7C0%7C637537547794353744%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=ThSNmb196ym4zG1rlajAZFEJKjfifWmte3jcDRedgic%3D&reserved=0> Music at the Gate: https://gbr01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.musicatthegate.ie%2F&data=04%7C01%7Ckieran.little%40northumberland.gov.uk%7C2d3d98b5a4d24a8c728108d8fd059a9e%7Cbb13a9de829042f0a980dc3bdfe70f40%7C0%7C0%7C637537547794353744%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=ouS43BZwF0m%2FAB8BUuRquDiI6aGOK7KOPmS6vMHROB8%3D&reserved=0<https://gbr01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.musicatthegate.ie%2F&data=04%7C01%7Ckieran.little%40northumberland.gov.uk%7C2d3d98b5a4d24a8c728108d8fd059a9e%7Cbb13a9de829042f0a980dc3bdfe70f40%7C0%7C0%7C637537547794353744%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=ouS43BZwF0m%2FAB8BUuRquDiI6aGOK7KOPmS6vMHROB8%3D&reserved=0> Ceol FM: https://gbr01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.ceol.fm%2F&data=04%7C01%7Ckieran.little%40northumberland.gov.uk%7C2d3d98b5a4d24a8c728108d8fd059a9e%7Cbb13a9de829042f0a980dc3bdfe70f40%7C0%7C0%7C637537547794353744%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=5ueHKhz1boa8QQOkSeG63gFQZt0tbxSZF%2BPYnr9c4vE%3D&reserved=0<https://gbr01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.ceol.fm%2F&data=04%7C01%7Ckieran.little%40northumberland.gov.uk%7C2d3d98b5a4d24a8c728108d8fd059a9e%7Cbb13a9de829042f0a980dc3bdfe70f40%7C0%7C0%7C637537547794353744%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=5ueHKhz1boa8QQOkSeG63gFQZt0tbxSZF%2BPYnr9c4vE%3D&reserved=0> Tel: 00353(0)877670464 Email: darragh@ceol.fm _______________________________________________ Blind-sysadmins mailing list -- blind-sysadmins@lists.hodgsonfamily.org To unsubscribe send an email to blind-sysadmins-leave@lists.hodgsonfamily.org Save Time Do It Online! We have made a few key improvements to our site to make our services easy to access. Now you can do everything from paying your council tax, to reporting a faulty street light online. Go to: www.northumberland.gov.uk and click 'pay, apply or report' to access the relevant forms. This email is intended solely for the individual or individuals to whom it is addressed, and may contain confidential and/or privileged material. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this email is prohibited. If you receive this email in error, please contact the sender and delete the email from any computer. All email communication may be subject to recording and/or monitoring in accordance with internal policy and relevant legislation. [Northumberland County Council Stay Home]
participants (6)
-
Andrew Hodgson -
Chris Nestrud -
Darragh Ó Héiligh -
Jason White -
John G Heim -
Kieran Little