let's talk Backup Exec
Hey everyone: I just got a promotion at work, and with the goodness of a promotion comes more responsibilities. I'm now in charge of our backup infrastructure. We run Backup Exec 12.5, and have 16 or so media servers in various branch offices around the world. While Backup Exec used to be very accessible, over the last two versions accessibility seems to have gone into something of a decline, I already use scripts just to read the main table of jobs, and might need more just to get around the scheduling window so I can actually schedule a job. I'd consider the rest of the program accessible but clunky, tabbing doesn't always bring you where you think it will, on some screens tab order seems to be dynamic. Backup Exec also has an odd way of bringing up alert windows and grabbing your focus, even when you tell it specifically not to. While clunkyness is OK on one server, when you have fifteen individual servers to manage and no way to do it in a centralized fashion clunkyness is less acceptable. This is also less than a quarter of my new responsibilities as well. The current admin has the remote desktop console setup with all the Backup Exec servers in his favorites, stays logged into each server with a Backup Exec window open, and flicks from screen to screen and can easily glance at the output from each job. For us screen reader users, that doesn't work well. I've already got e-mail alerts set up and am using some Outlook rules to sort failure jobs out from successes, but was wondering if there's anything else I can be doing? I'd think people in really large environments with hundreds of media servers aren't logging into each server on a daily basis, so there has to be better ways to handle this. Any ideas? Thanks. Ryan
Hay Ryan, What about using a CASO? This will allow you to use one instance of backup exec and then you'll see all the jobs, all the logs, all the media libraries and everything else in the one application. I also used a client to then connect to that CASO remotely so I didn't need to remote onto the server. I'm done with backup exec now fortunately. We've moved onto an enterprise level product called Data protector. It's almost even more inaccessible and instead of donig everything in one application, you've applications for the SAN, the EML, the VLS and then you've also a back end console for the data protecter UI as well because not all the functionality is available in the GUI. Fun times. Even funnier because I no longer have to do it full time. Shout if you have any questions I can help with. Regards Darragh Ó Héiligh Fujitsu Offices of the Houses of the Oireachtas, Fredrick Building, South Fredrick Street, Dublin2 Telephone: +353 (1) 618 3559 Email: darragh.oheiligh@oireachtas.ie Internet: http://www.oireachtas.ie From: Ryan Shugart <rshugart@pcisys.net> To: Blind sysadmins list <blind-sysadmins@lists.hodgsonfamily.org> Date: 16/06/2010 00:32 Subject: [Blind-sysadmins] let's talk Backup Exec Sent by: blind-sysadmins-bounces@lists.hodgsonfamily.org Hey everyone: I just got a promotion at work, and with the goodness of a promotion comes more responsibilities. I'm now in charge of our backup infrastructure. We run Backup Exec 12.5, and have 16 or so media servers in various branch offices around the world. While Backup Exec used to be very accessible, over the last two versions accessibility seems to have gone into something of a decline, I already use scripts just to read the main table of jobs, and might need more just to get around the scheduling window so I can actually schedule a job. I'd consider the rest of the program accessible but clunky, tabbing doesn't always bring you where you think it will, on some screens tab order seems to be dynamic. Backup Exec also has an odd way of bringing up alert windows and grabbing your focus, even when you tell it specifically not to. While clunkyness is OK on one server, when you have fifteen individual servers to manage and no way to do it in a centralized fashion clunkyness is less acceptable. This is also less than a quarter of my new responsibilities as well. The current admin has the remote desktop console setup with all the Backup Exec servers in his favorites, stays logged into each server with a Backup Exec window open, and flicks from screen to screen and can easily glance at the output from each job. For us screen reader users, that doesn't work well. I've already got e-mail alerts set up and am using some Outlook rules to sort failure jobs out from successes, but was wondering if there's anything else I can be doing? I'd think people in really large environments with hundreds of media servers aren't logging into each server on a daily basis, so there has to be better ways to handle this. Any ideas? Thanks. Ryan _______________________________________________ Blind-sysadmins mailing list Blind-sysadmins@lists.hodgsonfamily.org http://lists.hodgsonfamily.org/mailman/listinfo/blind-sysadmins
Hi Darragh: Thanks for the suggestion. Actually there's kind of a funny story there, we did have a CASO server up and running up until about a week ago for that purpose. Then something happened, and I say something because we still have no clue what happened or why it happened and Symantec's support is about as accessible as their product. Anyway, the CASO crashed and somehow brought down all the Backup Exec servers around the world with it, so it was decided above me that we won't be doing that again. I'm interested in migrating to Microsoft Data Protection Manager, it might be a little more robust than backup Exec assuming I get time to learn it and its accessible, but other than that I think we're sticking with what we have. In talking to others where I work, if there is a third party product that takes the place of the CASO we might still be interested in that. Thanks. Ryan -----Original Message----- From: blind-sysadmins-bounces@lists.hodgsonfamily.org [mailto:blind-sysadmins-bounces@lists.hodgsonfamily.org] On Behalf Of Darragh.OHeiligh@Oireachtas.ie Sent: Wednesday, June 16, 2010 2:23 AM To: Blind sysadmins list Cc: Blind sysadmins list; blind-sysadmins-bounces@lists.hodgsonfamily.org Subject: Re: [Blind-sysadmins] let's talk Backup Exec Hay Ryan, What about using a CASO? This will allow you to use one instance of backup exec and then you'll see all the jobs, all the logs, all the media libraries and everything else in the one application. I also used a client to then connect to that CASO remotely so I didn't need to remote onto the server. I'm done with backup exec now fortunately. We've moved onto an enterprise level product called Data protector. It's almost even more inaccessible and instead of donig everything in one application, you've applications for the SAN, the EML, the VLS and then you've also a back end console for the data protecter UI as well because not all the functionality is available in the GUI. Fun times. Even funnier because I no longer have to do it full time. Shout if you have any questions I can help with. Regards Darragh Ó Héiligh Fujitsu Offices of the Houses of the Oireachtas, Fredrick Building, South Fredrick Street, Dublin2 Telephone: +353 (1) 618 3559 Email: darragh.oheiligh@oireachtas.ie Internet: http://www.oireachtas.ie From: Ryan Shugart <rshugart@pcisys.net> To: Blind sysadmins list <blind-sysadmins@lists.hodgsonfamily.org> Date: 16/06/2010 00:32 Subject: [Blind-sysadmins] let's talk Backup Exec Sent by: blind-sysadmins-bounces@lists.hodgsonfamily.org Hey everyone: I just got a promotion at work, and with the goodness of a promotion comes more responsibilities. I'm now in charge of our backup infrastructure. We run Backup Exec 12.5, and have 16 or so media servers in various branch offices around the world. While Backup Exec used to be very accessible, over the last two versions accessibility seems to have gone into something of a decline, I already use scripts just to read the main table of jobs, and might need more just to get around the scheduling window so I can actually schedule a job. I'd consider the rest of the program accessible but clunky, tabbing doesn't always bring you where you think it will, on some screens tab order seems to be dynamic. Backup Exec also has an odd way of bringing up alert windows and grabbing your focus, even when you tell it specifically not to. While clunkyness is OK on one server, when you have fifteen individual servers to manage and no way to do it in a centralized fashion clunkyness is less acceptable. This is also less than a quarter of my new responsibilities as well. The current admin has the remote desktop console setup with all the Backup Exec servers in his favorites, stays logged into each server with a Backup Exec window open, and flicks from screen to screen and can easily glance at the output from each job. For us screen reader users, that doesn't work well. I've already got e-mail alerts set up and am using some Outlook rules to sort failure jobs out from successes, but was wondering if there's anything else I can be doing? I'd think people in really large environments with hundreds of media servers aren't logging into each server on a daily basis, so there has to be better ways to handle this. Any ideas? Thanks. Ryan _______________________________________________ Blind-sysadmins mailing list Blind-sysadmins@lists.hodgsonfamily.org http://lists.hodgsonfamily.org/mailman/listinfo/blind-sysadmins _______________________________________________ Blind-sysadmins mailing list Blind-sysadmins@lists.hodgsonfamily.org http://lists.hodgsonfamily.org/mailman/listinfo/blind-sysadmins
Good morning, I've asked about this a long time ago but I didn't really follow it up. Our environment has doubled in size in terms of servers over the past year. We now have two DR sites, I'm in the process of virtualizing our DMZ and we're expanding our SQL cluster. The amount of work that has been done is actually quite impressive if I do say so myself. The problem is that there's just too much information to digest every morning. I've syslog showing errors, What's up gold showing utilization and availability, SCOM showing system errors, System insight manager monitoring the SAN, Storage escential checking for storage bottle necks, OfficeScan monitoring for viruses and other infections and Nessus running security reports. It all mounts up to a huge amount of reports and statistics to monitor every day. The problem is, if I get distracted by OfficeScan for example, the other reports are neglected and I potentially miss things that have happened during the night. Of course, the other person that works with me can glance up at the monitoring screens and see at a glance what's up and what's not. He can see systems that are running low on disk space or using up far too much memory. I start an hour before this person so it looks terrible if I miss something that can be spotted by simply looking up at the screens. How do you monitor hundreds of servers? Are there any tips or tricks you'd like to share? We're working with a mixed environment here but if I need to use two approaches for monitoring both widnows and Linux then I don't mind. Once I can get information in a more condensed format without overwelming me with things I don't need to know about. Thanks. any suggestions will be appreciated. Regards Darragh Ó Héiligh Fujitsu Offices of the Houses of the Oireachtas, Fredrick Building, South Fredrick Street, Dublin2 Telephone: +353 (1) 618 3559 Email: darragh.oheiligh@oireachtas.ie Internet: http://www.oireachtas.ie
Good morning, I've asked about this a long time ago but I didn't really follow it up. Our environment has doubled in size in terms of servers over the past year. We now have two DR sites, I'm in the process of virtualizing our DMZ and we're expanding our SQL cluster. The amount of work that has been done is actually quite impressive if I do say so myself. The problem is that there's just too much information to digest every morning. I've syslog showing errors, What's up gold showing utilization and availability, SCOM showing system errors, System insight manager monitoring the SAN, Storage escential checking for storage bottle necks, OfficeScan monitoring for viruses and other infections and Nessus running security reports. It all mounts up to a huge amount of reports and statistics to monitor every day. The problem is, if I get distracted by OfficeScan for example, the other reports are neglected and I potentially miss things that have happened during the night. Of course, the other person that works with me can glance up at the monitoring screens and see at a glance what's up and what's not. He can see systems that are running low on disk space or using up far too much memory. I start an hour before this person so it looks terrible if I miss something that can be spotted by simply looking up at the screens. How do you monitor hundreds of servers? Are there any tips or tricks you'd like to share? We're working with a mixed environment here but if I need to use two approaches for monitoring both widnows and Linux then I don't mind. Once I can get information in a more condensed format without overwelming me with things I don't need to know about. Thanks. any suggestions will be appreciated. Regards Darragh Ó Héiligh Fujitsu Offices of the Houses of the Oireachtas, Fredrick Building, South Fredrick Street, Dublin2 Telephone: +353 (1) 618 3559 Email: darragh.oheiligh@oireachtas.ie Internet: http://www.oireachtas.ie
Hi, I use email alerting as a primary method to get this type of information, and am also very hot on making sure that if I don't need to know about it, that the alerts are tailored so that inboxes are not cluttered up. We probably have a smaller environment than you (around 100 servers), but I know where you are coming from. We do also have a monitoring screen, and I had to get this set up on the request of the business, who wanted something graphical that people could see. Getting that exactly right is an art in itself, as they wanted people to be able to glance at business transactional data to ensure that we were still processing transactions on various payment websites. That meant we had to craft our own monitoring scripts to get at the information we needed. Andrew. -----Original Message----- From: blind-sysadmins-bounces@lists.hodgsonfamily.org [mailto:blind-sysadmins-bounces@lists.hodgsonfamily.org] On Behalf Of Darragh OHeiligh Sent: 20 December 2011 09:05 To: Blind sysadmins list Subject: [Blind-sysadmins] Keeping track of your environment. Good morning, I've asked about this a long time ago but I didn't really follow it up. Our environment has doubled in size in terms of servers over the past year. We now have two DR sites, I'm in the process of virtualizing our DMZ and we're expanding our SQL cluster. The amount of work that has been done is actually quite impressive if I do say so myself. The problem is that there's just too much information to digest every morning. I've syslog showing errors, What's up gold showing utilization and availability, SCOM showing system errors, System insight manager monitoring the SAN, Storage escential checking for storage bottle necks, OfficeScan monitoring for viruses and other infections and Nessus running security reports. It all mounts up to a huge amount of reports and statistics to monitor every day. The problem is, if I get distracted by OfficeScan for example, the other reports are neglected and I potentially miss things that have happened during the night. Of course, the other person that works with me can glance up at the monitoring screens and see at a glance what's up and what's not. He can see systems that are running low on disk space or using up far too much memory. I start an hour before this person so it looks terrible if I miss something that can be spotted by simply looking up at the screens. How do you monitor hundreds of servers? Are there any tips or tricks you'd like to share? We're working with a mixed environment here but if I need to use two approaches for monitoring both widnows and Linux then I don't mind. Once I can get information in a more condensed format without overwelming me with things I don't need to know about. Thanks. any suggestions will be appreciated. Regards Darragh Ó Héiligh Fujitsu Offices of the Houses of the Oireachtas, Fredrick Building, South Fredrick Street, Dublin2 Telephone: +353 (1) 618 3559 Email: darragh.oheiligh@oireachtas.ie Internet: http://www.oireachtas.ie _______________________________________________ Blind-sysadmins mailing list Blind-sysadmins@lists.hodgsonfamily.org http://lists.hodgsonfamily.org/listinfo/blind-sysadmins
Hi, I use email alerting as a primary method to get this type of information, and am also very hot on making sure that if I don't need to know about it, that the alerts are tailored so that inboxes are not cluttered up. We probably have a smaller environment than you (around 100 servers), but I know where you are coming from. We do also have a monitoring screen, and I had to get this set up on the request of the business, who wanted something graphical that people could see. Getting that exactly right is an art in itself, as they wanted people to be able to glance at business transactional data to ensure that we were still processing transactions on various payment websites. That meant we had to craft our own monitoring scripts to get at the information we needed. Andrew. -----Original Message----- From: blind-sysadmins-bounces@lists.hodgsonfamily.org [mailto:blind-sysadmins-bounces@lists.hodgsonfamily.org] On Behalf Of Darragh OHeiligh Sent: 20 December 2011 09:05 To: Blind sysadmins list Subject: [Blind-sysadmins] Keeping track of your environment. Good morning, I've asked about this a long time ago but I didn't really follow it up. Our environment has doubled in size in terms of servers over the past year. We now have two DR sites, I'm in the process of virtualizing our DMZ and we're expanding our SQL cluster. The amount of work that has been done is actually quite impressive if I do say so myself. The problem is that there's just too much information to digest every morning. I've syslog showing errors, What's up gold showing utilization and availability, SCOM showing system errors, System insight manager monitoring the SAN, Storage escential checking for storage bottle necks, OfficeScan monitoring for viruses and other infections and Nessus running security reports. It all mounts up to a huge amount of reports and statistics to monitor every day. The problem is, if I get distracted by OfficeScan for example, the other reports are neglected and I potentially miss things that have happened during the night. Of course, the other person that works with me can glance up at the monitoring screens and see at a glance what's up and what's not. He can see systems that are running low on disk space or using up far too much memory. I start an hour before this person so it looks terrible if I miss something that can be spotted by simply looking up at the screens. How do you monitor hundreds of servers? Are there any tips or tricks you'd like to share? We're working with a mixed environment here but if I need to use two approaches for monitoring both widnows and Linux then I don't mind. Once I can get information in a more condensed format without overwelming me with things I don't need to know about. Thanks. any suggestions will be appreciated. Regards Darragh Ó Héiligh Fujitsu Offices of the Houses of the Oireachtas, Fredrick Building, South Fredrick Street, Dublin2 Telephone: +353 (1) 618 3559 Email: darragh.oheiligh@oireachtas.ie Internet: http://www.oireachtas.ie _______________________________________________ Blind-sysadmins mailing list Blind-sysadmins@lists.hodgsonfamily.org http://lists.hodgsonfamily.org/listinfo/blind-sysadmins
Hi Darragh: Can you go into more detail on what these "monitor screens" are? It sounds like a billboard hanging on a wall displaying system status. We don't have anything like that in my environment. We have a pager that rotates through the department each week, and our monitoring software is set to flag critical alerts and send those off to the person on call and they'll be responsible for finding the right party and informing them. Beyond that, everyone's kind of responsible for monitoring their own systems. I really have to pitty our backup guy, that used to be me but that got too much, and its getting too much for the current backup guy too. I agree with Andrew here, its all about using whatever software you have's monitoring and alerting, but really narrowing down just what alerts you need. If its something I need to know about right away, that's one type of alert. If its something that can wait for the next business day, but needs to be top on my plate come next business day then that's something else. Also I just skim systems from time to time to see if anything is slipping through the cracks. Is it perfect, no, but what is? Good monitoring is actually very difficult, sighted or blind. We recently spent about $250000 on a package called EG, the vendor made our management believe EG would not just alert us to systems being down, but it would also automatically do route cause analisys and find the real cause of problems. Surprisingly it hasn't worked too well, I only discovered a bad SAN drive today when I wandered into the server room and heard a really odd high pitched squeal coming from the SAN. No other alerts period. So getting the information is tough regardless, and it just boils down to good tweaking. On a side note Darragh, I'd be very interested in talking to you further privately about your setup. It sounds like we manage roughly the same types of environments, and I'd love to hear more about what challenges you face and the things you've done to get around them. For example, I didn't find SCOM to be that accessible, nothing like SCCM. I'd love to set up SCOM to monitor SCCM, right now our SCCM environment is completely unmonitored, well, accept for me browsing sight status from time to time. Ryan -----Original Message----- From: blind-sysadmins-bounces@lists.hodgsonfamily.org [mailto:blind-sysadmins-bounces@lists.hodgsonfamily.org] On Behalf Of Darragh OHeiligh Sent: Tuesday, December 20, 2011 2:05 AM To: Blind sysadmins list Subject: [Blind-sysadmins] Keeping track of your environment. Good morning, I've asked about this a long time ago but I didn't really follow it up. Our environment has doubled in size in terms of servers over the past year. We now have two DR sites, I'm in the process of virtualizing our DMZ and we're expanding our SQL cluster. The amount of work that has been done is actually quite impressive if I do say so myself. The problem is that there's just too much information to digest every morning. I've syslog showing errors, What's up gold showing utilization and availability, SCOM showing system errors, System insight manager monitoring the SAN, Storage escential checking for storage bottle necks, OfficeScan monitoring for viruses and other infections and Nessus running security reports. It all mounts up to a huge amount of reports and statistics to monitor every day. The problem is, if I get distracted by OfficeScan for example, the other reports are neglected and I potentially miss things that have happened during the night. Of course, the other person that works with me can glance up at the monitoring screens and see at a glance what's up and what's not. He can see systems that are running low on disk space or using up far too much memory. I start an hour before this person so it looks terrible if I miss something that can be spotted by simply looking up at the screens. How do you monitor hundreds of servers? Are there any tips or tricks you'd like to share? We're working with a mixed environment here but if I need to use two approaches for monitoring both widnows and Linux then I don't mind. Once I can get information in a more condensed format without overwelming me with things I don't need to know about. Thanks. any suggestions will be appreciated. Regards Darragh Ó Héiligh Fujitsu Offices of the Houses of the Oireachtas, Fredrick Building, South Fredrick Street, Dublin2 Telephone: +353 (1) 618 3559 Email: darragh.oheiligh@oireachtas.ie Internet: http://www.oireachtas.ie _______________________________________________ Blind-sysadmins mailing list Blind-sysadmins@lists.hodgsonfamily.org http://lists.hodgsonfamily.org/listinfo/blind-sysadmins
We use nagios to monitor our systems The idea behind nagios is that you write a script that takes arguments for critical and warning error levels. So the critical level can be above or below a certain number or even not matching a string pattern. Then the script can fetch a value via any method you like and do the comparison. It then sends you an alert if the critical or warning levels are met. Of course, there are a huge number of existing scripts you can use. We use nagios to check everything from toner level in our printers to disk space on our mail server. Mostly, we use snmp to fetch the data for comparison but you can use anything, http, soap, dns. And then you can configure just about any kind of alert you like. It doesn't have to be email. I gave a demonstration on configuring nagios to my local linux users group and set it up to call my cell phone and speak an error message. So during the demo, I got a call on my cell phone and I held it up so the people in the front row could hear it say in the festival voice that my file server was down. Here is a link to the web page I used when giving my talk: http://www.math.wisc.edu/~jheim/snmp/ ----- Original Message ----- From: "Ryan Shugart" <rshugart@pcisys.net> To: "Blind sysadmins list" <blind-sysadmins@lists.hodgsonfamily.org> Sent: Tuesday, December 20, 2011 6:58 PM Subject: Re: [Blind-sysadmins] Keeping track of your environment. Hi Darragh: Can you go into more detail on what these "monitor screens" are? It sounds like a billboard hanging on a wall displaying system status. We don't have anything like that in my environment. We have a pager that rotates through the department each week, and our monitoring software is set to flag critical alerts and send those off to the person on call and they'll be responsible for finding the right party and informing them. Beyond that, everyone's kind of responsible for monitoring their own systems. I really have to pitty our backup guy, that used to be me but that got too much, and its getting too much for the current backup guy too. I agree with Andrew here, its all about using whatever software you have's monitoring and alerting, but really narrowing down just what alerts you need. If its something I need to know about right away, that's one type of alert. If its something that can wait for the next business day, but needs to be top on my plate come next business day then that's something else. Also I just skim systems from time to time to see if anything is slipping through the cracks. Is it perfect, no, but what is? Good monitoring is actually very difficult, sighted or blind. We recently spent about $250000 on a package called EG, the vendor made our management believe EG would not just alert us to systems being down, but it would also automatically do route cause analisys and find the real cause of problems. Surprisingly it hasn't worked too well, I only discovered a bad SAN drive today when I wandered into the server room and heard a really odd high pitched squeal coming from the SAN. No other alerts period. So getting the information is tough regardless, and it just boils down to good tweaking. On a side note Darragh, I'd be very interested in talking to you further privately about your setup. It sounds like we manage roughly the same types of environments, and I'd love to hear more about what challenges you face and the things you've done to get around them. For example, I didn't find SCOM to be that accessible, nothing like SCCM. I'd love to set up SCOM to monitor SCCM, right now our SCCM environment is completely unmonitored, well, accept for me browsing sight status from time to time. Ryan -----Original Message----- From: blind-sysadmins-bounces@lists.hodgsonfamily.org [mailto:blind-sysadmins-bounces@lists.hodgsonfamily.org] On Behalf Of Darragh OHeiligh Sent: Tuesday, December 20, 2011 2:05 AM To: Blind sysadmins list Subject: [Blind-sysadmins] Keeping track of your environment. Good morning, I've asked about this a long time ago but I didn't really follow it up. Our environment has doubled in size in terms of servers over the past year. We now have two DR sites, I'm in the process of virtualizing our DMZ and we're expanding our SQL cluster. The amount of work that has been done is actually quite impressive if I do say so myself. The problem is that there's just too much information to digest every morning. I've syslog showing errors, What's up gold showing utilization and availability, SCOM showing system errors, System insight manager monitoring the SAN, Storage escential checking for storage bottle necks, OfficeScan monitoring for viruses and other infections and Nessus running security reports. It all mounts up to a huge amount of reports and statistics to monitor every day. The problem is, if I get distracted by OfficeScan for example, the other reports are neglected and I potentially miss things that have happened during the night. Of course, the other person that works with me can glance up at the monitoring screens and see at a glance what's up and what's not. He can see systems that are running low on disk space or using up far too much memory. I start an hour before this person so it looks terrible if I miss something that can be spotted by simply looking up at the screens. How do you monitor hundreds of servers? Are there any tips or tricks you'd like to share? We're working with a mixed environment here but if I need to use two approaches for monitoring both widnows and Linux then I don't mind. Once I can get information in a more condensed format without overwelming me with things I don't need to know about. Thanks. any suggestions will be appreciated. Regards Darragh Ó Héiligh Fujitsu Offices of the Houses of the Oireachtas, Fredrick Building, South Fredrick Street, Dublin2 Telephone: +353 (1) 618 3559 Email: darragh.oheiligh@oireachtas.ie Internet: http://www.oireachtas.ie _______________________________________________ Blind-sysadmins mailing list Blind-sysadmins@lists.hodgsonfamily.org http://lists.hodgsonfamily.org/listinfo/blind-sysadmins _______________________________________________ Blind-sysadmins mailing list Blind-sysadmins@lists.hodgsonfamily.org http://lists.hodgsonfamily.org/listinfo/blind-sysadmins
Hi Darragh: Can you go into more detail on what these "monitor screens" are? It sounds like a billboard hanging on a wall displaying system status. We don't have anything like that in my environment. We have a pager that rotates through the department each week, and our monitoring software is set to flag critical alerts and send those off to the person on call and they'll be responsible for finding the right party and informing them. Beyond that, everyone's kind of responsible for monitoring their own systems. I really have to pitty our backup guy, that used to be me but that got too much, and its getting too much for the current backup guy too. I agree with Andrew here, its all about using whatever software you have's monitoring and alerting, but really narrowing down just what alerts you need. If its something I need to know about right away, that's one type of alert. If its something that can wait for the next business day, but needs to be top on my plate come next business day then that's something else. Also I just skim systems from time to time to see if anything is slipping through the cracks. Is it perfect, no, but what is? Good monitoring is actually very difficult, sighted or blind. We recently spent about $250000 on a package called EG, the vendor made our management believe EG would not just alert us to systems being down, but it would also automatically do route cause analisys and find the real cause of problems. Surprisingly it hasn't worked too well, I only discovered a bad SAN drive today when I wandered into the server room and heard a really odd high pitched squeal coming from the SAN. No other alerts period. So getting the information is tough regardless, and it just boils down to good tweaking. On a side note Darragh, I'd be very interested in talking to you further privately about your setup. It sounds like we manage roughly the same types of environments, and I'd love to hear more about what challenges you face and the things you've done to get around them. For example, I didn't find SCOM to be that accessible, nothing like SCCM. I'd love to set up SCOM to monitor SCCM, right now our SCCM environment is completely unmonitored, well, accept for me browsing sight status from time to time. Ryan -----Original Message----- From: blind-sysadmins-bounces@lists.hodgsonfamily.org [mailto:blind-sysadmins-bounces@lists.hodgsonfamily.org] On Behalf Of Darragh OHeiligh Sent: Tuesday, December 20, 2011 2:05 AM To: Blind sysadmins list Subject: [Blind-sysadmins] Keeping track of your environment. Good morning, I've asked about this a long time ago but I didn't really follow it up. Our environment has doubled in size in terms of servers over the past year. We now have two DR sites, I'm in the process of virtualizing our DMZ and we're expanding our SQL cluster. The amount of work that has been done is actually quite impressive if I do say so myself. The problem is that there's just too much information to digest every morning. I've syslog showing errors, What's up gold showing utilization and availability, SCOM showing system errors, System insight manager monitoring the SAN, Storage escential checking for storage bottle necks, OfficeScan monitoring for viruses and other infections and Nessus running security reports. It all mounts up to a huge amount of reports and statistics to monitor every day. The problem is, if I get distracted by OfficeScan for example, the other reports are neglected and I potentially miss things that have happened during the night. Of course, the other person that works with me can glance up at the monitoring screens and see at a glance what's up and what's not. He can see systems that are running low on disk space or using up far too much memory. I start an hour before this person so it looks terrible if I miss something that can be spotted by simply looking up at the screens. How do you monitor hundreds of servers? Are there any tips or tricks you'd like to share? We're working with a mixed environment here but if I need to use two approaches for monitoring both widnows and Linux then I don't mind. Once I can get information in a more condensed format without overwelming me with things I don't need to know about. Thanks. any suggestions will be appreciated. Regards Darragh Ó Héiligh Fujitsu Offices of the Houses of the Oireachtas, Fredrick Building, South Fredrick Street, Dublin2 Telephone: +353 (1) 618 3559 Email: darragh.oheiligh@oireachtas.ie Internet: http://www.oireachtas.ie _______________________________________________ Blind-sysadmins mailing list Blind-sysadmins@lists.hodgsonfamily.org http://lists.hodgsonfamily.org/listinfo/blind-sysadmins
Hi Darragh: Thanks for the suggestion. Actually there's kind of a funny story there, we did have a CASO server up and running up until about a week ago for that purpose. Then something happened, and I say something because we still have no clue what happened or why it happened and Symantec's support is about as accessible as their product. Anyway, the CASO crashed and somehow brought down all the Backup Exec servers around the world with it, so it was decided above me that we won't be doing that again. I'm interested in migrating to Microsoft Data Protection Manager, it might be a little more robust than backup Exec assuming I get time to learn it and its accessible, but other than that I think we're sticking with what we have. In talking to others where I work, if there is a third party product that takes the place of the CASO we might still be interested in that. Thanks. Ryan -----Original Message----- From: blind-sysadmins-bounces@lists.hodgsonfamily.org [mailto:blind-sysadmins-bounces@lists.hodgsonfamily.org] On Behalf Of Darragh.OHeiligh@Oireachtas.ie Sent: Wednesday, June 16, 2010 2:23 AM To: Blind sysadmins list Cc: Blind sysadmins list; blind-sysadmins-bounces@lists.hodgsonfamily.org Subject: Re: [Blind-sysadmins] let's talk Backup Exec Hay Ryan, What about using a CASO? This will allow you to use one instance of backup exec and then you'll see all the jobs, all the logs, all the media libraries and everything else in the one application. I also used a client to then connect to that CASO remotely so I didn't need to remote onto the server. I'm done with backup exec now fortunately. We've moved onto an enterprise level product called Data protector. It's almost even more inaccessible and instead of donig everything in one application, you've applications for the SAN, the EML, the VLS and then you've also a back end console for the data protecter UI as well because not all the functionality is available in the GUI. Fun times. Even funnier because I no longer have to do it full time. Shout if you have any questions I can help with. Regards Darragh Ó Héiligh Fujitsu Offices of the Houses of the Oireachtas, Fredrick Building, South Fredrick Street, Dublin2 Telephone: +353 (1) 618 3559 Email: darragh.oheiligh@oireachtas.ie Internet: http://www.oireachtas.ie From: Ryan Shugart <rshugart@pcisys.net> To: Blind sysadmins list <blind-sysadmins@lists.hodgsonfamily.org> Date: 16/06/2010 00:32 Subject: [Blind-sysadmins] let's talk Backup Exec Sent by: blind-sysadmins-bounces@lists.hodgsonfamily.org Hey everyone: I just got a promotion at work, and with the goodness of a promotion comes more responsibilities. I'm now in charge of our backup infrastructure. We run Backup Exec 12.5, and have 16 or so media servers in various branch offices around the world. While Backup Exec used to be very accessible, over the last two versions accessibility seems to have gone into something of a decline, I already use scripts just to read the main table of jobs, and might need more just to get around the scheduling window so I can actually schedule a job. I'd consider the rest of the program accessible but clunky, tabbing doesn't always bring you where you think it will, on some screens tab order seems to be dynamic. Backup Exec also has an odd way of bringing up alert windows and grabbing your focus, even when you tell it specifically not to. While clunkyness is OK on one server, when you have fifteen individual servers to manage and no way to do it in a centralized fashion clunkyness is less acceptable. This is also less than a quarter of my new responsibilities as well. The current admin has the remote desktop console setup with all the Backup Exec servers in his favorites, stays logged into each server with a Backup Exec window open, and flicks from screen to screen and can easily glance at the output from each job. For us screen reader users, that doesn't work well. I've already got e-mail alerts set up and am using some Outlook rules to sort failure jobs out from successes, but was wondering if there's anything else I can be doing? I'd think people in really large environments with hundreds of media servers aren't logging into each server on a daily basis, so there has to be better ways to handle this. Any ideas? Thanks. Ryan _______________________________________________ Blind-sysadmins mailing list Blind-sysadmins@lists.hodgsonfamily.org http://lists.hodgsonfamily.org/mailman/listinfo/blind-sysadmins _______________________________________________ Blind-sysadmins mailing list Blind-sysadmins@lists.hodgsonfamily.org http://lists.hodgsonfamily.org/mailman/listinfo/blind-sysadmins
Hay Ryan, What about using a CASO? This will allow you to use one instance of backup exec and then you'll see all the jobs, all the logs, all the media libraries and everything else in the one application. I also used a client to then connect to that CASO remotely so I didn't need to remote onto the server. I'm done with backup exec now fortunately. We've moved onto an enterprise level product called Data protector. It's almost even more inaccessible and instead of donig everything in one application, you've applications for the SAN, the EML, the VLS and then you've also a back end console for the data protecter UI as well because not all the functionality is available in the GUI. Fun times. Even funnier because I no longer have to do it full time. Shout if you have any questions I can help with. Regards Darragh Ó Héiligh Fujitsu Offices of the Houses of the Oireachtas, Fredrick Building, South Fredrick Street, Dublin2 Telephone: +353 (1) 618 3559 Email: darragh.oheiligh@oireachtas.ie Internet: http://www.oireachtas.ie From: Ryan Shugart <rshugart@pcisys.net> To: Blind sysadmins list <blind-sysadmins@lists.hodgsonfamily.org> Date: 16/06/2010 00:32 Subject: [Blind-sysadmins] let's talk Backup Exec Sent by: blind-sysadmins-bounces@lists.hodgsonfamily.org Hey everyone: I just got a promotion at work, and with the goodness of a promotion comes more responsibilities. I'm now in charge of our backup infrastructure. We run Backup Exec 12.5, and have 16 or so media servers in various branch offices around the world. While Backup Exec used to be very accessible, over the last two versions accessibility seems to have gone into something of a decline, I already use scripts just to read the main table of jobs, and might need more just to get around the scheduling window so I can actually schedule a job. I'd consider the rest of the program accessible but clunky, tabbing doesn't always bring you where you think it will, on some screens tab order seems to be dynamic. Backup Exec also has an odd way of bringing up alert windows and grabbing your focus, even when you tell it specifically not to. While clunkyness is OK on one server, when you have fifteen individual servers to manage and no way to do it in a centralized fashion clunkyness is less acceptable. This is also less than a quarter of my new responsibilities as well. The current admin has the remote desktop console setup with all the Backup Exec servers in his favorites, stays logged into each server with a Backup Exec window open, and flicks from screen to screen and can easily glance at the output from each job. For us screen reader users, that doesn't work well. I've already got e-mail alerts set up and am using some Outlook rules to sort failure jobs out from successes, but was wondering if there's anything else I can be doing? I'd think people in really large environments with hundreds of media servers aren't logging into each server on a daily basis, so there has to be better ways to handle this. Any ideas? Thanks. Ryan _______________________________________________ Blind-sysadmins mailing list Blind-sysadmins@lists.hodgsonfamily.org http://lists.hodgsonfamily.org/mailman/listinfo/blind-sysadmins
Ryan, Good work in getting the promotion. does the backup tool have any COM/DOM where you can control the app via scripts? such as VBA or Powershell? Then you could create scripts to do the sceduling and even possibily improve accessibility via scripts with Jaws or Window-eyes. Sean
Hi Sean: Thanks a lot for the congradulations. I don't think Backup Exec has any kind of API, a com object is possible I've not looked for that yet. What it does have though that might be just as good is a commandline tool called bemcmd.exe. I've looked at the help for this tool, and while it looks like its kind of bulky, meaning I wouldn't want to use it for day to day administration and I don't think that's how Symantec designed it to be used, if I need screen reader scripts written this might be the way to go, instead of working with the GUI itself, having a custom GUI that calls bemcmd.exe. I should know a little more today as I get to go through this for the first time and explore. Thanks again Ryan -----Original Message----- From: blind-sysadmins-bounces@lists.hodgsonfamily.org [mailto:blind-sysadmins-bounces@lists.hodgsonfamily.org] On Behalf Of Sean Murphy Sent: Thursday, June 17, 2010 5:12 AM To: Blind sysadmins list Subject: Re: [Blind-sysadmins] let's talk Backup Exec Ryan, Good work in getting the promotion. does the backup tool have any COM/DOM where you can control the app via scripts? such as VBA or Powershell? Then you could create scripts to do the sceduling and even possibily improve accessibility via scripts with Jaws or Window-eyes. Sean _______________________________________________ Blind-sysadmins mailing list Blind-sysadmins@lists.hodgsonfamily.org http://lists.hodgsonfamily.org/mailman/listinfo/blind-sysadmins
Hi Sean: Thanks a lot for the congradulations. I don't think Backup Exec has any kind of API, a com object is possible I've not looked for that yet. What it does have though that might be just as good is a commandline tool called bemcmd.exe. I've looked at the help for this tool, and while it looks like its kind of bulky, meaning I wouldn't want to use it for day to day administration and I don't think that's how Symantec designed it to be used, if I need screen reader scripts written this might be the way to go, instead of working with the GUI itself, having a custom GUI that calls bemcmd.exe. I should know a little more today as I get to go through this for the first time and explore. Thanks again Ryan -----Original Message----- From: blind-sysadmins-bounces@lists.hodgsonfamily.org [mailto:blind-sysadmins-bounces@lists.hodgsonfamily.org] On Behalf Of Sean Murphy Sent: Thursday, June 17, 2010 5:12 AM To: Blind sysadmins list Subject: Re: [Blind-sysadmins] let's talk Backup Exec Ryan, Good work in getting the promotion. does the backup tool have any COM/DOM where you can control the app via scripts? such as VBA or Powershell? Then you could create scripts to do the sceduling and even possibily improve accessibility via scripts with Jaws or Window-eyes. Sean _______________________________________________ Blind-sysadmins mailing list Blind-sysadmins@lists.hodgsonfamily.org http://lists.hodgsonfamily.org/mailman/listinfo/blind-sysadmins
participants (6)
-
Andrew Hodgson -
Darragh OHeiligh -
Darragh.OHeiligh＠Oireachtas.ie
-
John G. Heim -
Ryan Shugart -
Sean Murphy