Re: [Blind-sysadmins] Custom server monitor application.

Last year we started using SCOM just to monitor our web sites. We use EG for everything else but since our web sites are all ASP.Net based it made more sense, and we’ve been really happy with it. One really cool thing is we set up a feature (I forgot the exact name for it) where in addition to SCOM running tests locally, it also runs tests on our web sites from Windows Azure datacenter around the world. That way we can tell, say, what’s the responsiveness of our sites for a user in Asia somewhere, for example. I think that might be a bit beyond what Darragh is looking for, but as we’re a global company it is really neat and the web team really likes the data. We also have another product tied in called Citratest that simulates a human at a computer performing actions against various applications, and we use that to test our web sites and even some stuff with our ERP application, AKA Citratest simulates the process of someone placing an order. Again way beyond what Darragh needs, but pretty cool. I don’t play with most of that, so no clue how accessible it’d be. Ryan

On Nov 24, 2015, at 8:18 AM, Andrew Hodgson wrote:

Hi,

I actually ended up cooking our own monitoring in my last place since there were specific business impact things we needed to monitor that couldn't be monitored well with most solutions. Having said this, I found up.time a good monitoring tool if people want to go with a packaged version. We also used AppDynamics, which is a fab tool if you're a .net house with lots of .net websites, and need to monitor what is happening on a low .net level.

Andrew. ________________________________________ From: Blind-sysadmins [blind-sysadmins-bounces@lists.hodgsonfamily.org] on behalf of Ryan Shugart [rshugart@ryanshugart.com] Sent: 24 November 2015 15:08 To: Blind sysadmins list Subject: Re: [Blind-sysadmins] Custom server monitor application.

Hi Darragh: Your reasoning makes sense, and agreed Nagios is a lot to learn. I actually don’t know it that well, the place where I started used an app called KSHostmon, which was really accessible at the time (don’t know if it still is) and did most of what we needed. We use EG where I am now, and I think its horrible. Its all web UI based with no accessibility at all, and from a non accessibility standpoint making things like dependencies work properly is difficult, and it sometimes just fails. There are actually 7 servers just doing various monitoring tasks, and granted they monitor over 1000 servers but still. EG says we should have another server at each remote site, which we won’t do because the licensing is very expensive, I think like $1500 per remote agent. So no I would not recommend EG. Ryan

...

On Nov 24, 2015, at 4:26 AM, Darragh Ó Héiligh wrote:

Bill and Ryan, thanks for your responses.

Bill, I'll try this later. I'd love to get this workin.

Ryan, I love the idea of creating custom actions. I'm not quite sure yet how I'll do this without really complicating the application or creating a graphical front end but it's going to be a nice challenge. I also like the idea of sending text notifications. I was actually thinking of this yesterday. I'll add an option to specify a list of phone numbers to be notified. If a SMS should be sent, I'll format a smaller message within 161 characters that's more specific to a phone user.

You're absolutely right. I'm reinventing the wheel as Whats up gold, SCOM and Nageos can do all of this however. 1. Whatsup gold is quite expensive. 2. SCOM is very expensive. 3. I'd love to invest time in learning Nageos. I was making some progress with it years ago but the company who maintained this environment before I got here implemented Nageos but didn't document any of it. I haven't enough knowledge of the system to piece together what they've done. Also, they half finished a lot of the monitors so things only half work. We also get no notifications.

My thought is that this small application will do most of the monitoring that I'll need.

I se the question quite often on different lists. How does someone monitor small environments. Well, this could be a nice option for some.

I think you have good experience with using Nageos. If you'd ever like to impart some of that knowledge I'd be very interested. -----Original Message----- From: Blind-sysadmins [mailto:blind-sysadmins-bounces@lists.hodgsonfamily.org] On Behalf Of Bill Dengler Sent: Tuesday 24 November 2015 03:50 To: Blind sysadmins list Subject: Re: [Blind-sysadmins] Custom server monitor application.

For push notifications to iOS devices, you could use Boxcar (http://boxcar.io http://boxcar.io/). They have an HTTP API, and I wrote Python bindings for it if you’re interested.

Bill

...

On Nov 23, 2015, at 10:37 PM, Ryan Shugart wrote:

Hi Darragh: A couple thoughts, in addition to sending an email, sending out a text message or perhaps a push notification to mobile devices may be an option. I’m not sure how you’re developing the app and each service (Google, Apple, MS) have their own hooks into push notifications, so that may not be possible. The other thing to perhaps consider adding, as a remediation action to pretty much any of these issues, perhaps allowing the user to invoke a set Powershell script when an alarm is triggered, that would provide for some flexibility, sometimes there’s a specific way to start a service or such, and that lets the user say to run powershell script X when the situation hits. I guess my final question would be what is your goal in developing this tool? There’s several other applications out there that do this and much more, obviously SCOM being the 800 LB gorilla, but Nagios, EG, Whatsup and several others do this, you could also probably just write Powershell scripts to do much of it using background tasks and such. So what do you want to bring to the table? Ryan

...

On Nov 23, 2015, at 1:30 PM, Darragh Ó Héiligh wrote:

I'm writing an application at the moment. It started as a thing that I really needed but it's evolving into something a bit more generic.

I'm looking for suggestions on useful ways that it could be developed further to make it useful.

It's primary purpose is as a monitoring tool. It isn't meant as a data analytics utility.

It's current functions are:

1. Restart services. These services can either be given fully in the application config or a number of service prifixes can be used. For example: If I want to restart all report services on an application server and I know these services start with the letters RPT I can restart them with a minimum of configuration.

2. Check services. If a service is set to start up automatically but it's currently stopped, alarm bells should wring. The application Emails an address or list of addresses in the application config when services are not started. The SMTP host, sender address and port are also configurable. This check can also be configured to start any services that are not running but should be.

3. Delete temp files from specific directories. Again, specifically useful for application servers. Let's say you have a folder called SBS with dozens of temp sub folders with files ending in .xml, .pdf, .csv, .cab etc. This will go and delete everything in there ending with these extensions. Of course, these extensions are configurable in... You've guessed it, the application config. However, let's say you have a sub folder within your temp folder called DoNotTouch or Services or something like that. An exclusion or list of exclusions can be added to the application.config so that these folders won't be touched.

4. Log archiving. There are applications that handle log role overs really well but there are also applications that just write to the one log file for years. This becomes impossible to open never mind read! Using the application config you can specify the base folder that these are found in then a name or partial name so that the application will recursively look through and move all of these log files to an archive location. This location is set in the application config but it's also dated and in a folder corresponding to the server name that the application is run on to allow for use on multiple servers connecting to the one NAS for example.

5. Check utilization. Again, this is configurable. Set the threshold of percent free space or percent RAM or CPU available using the application config and the application will Email you and write to the event log when this threshold is reached.

6. Of course there's also a debug mode that will show you what it will do with each function without actually performing any action. There's also a help list so you can check the available options right from the command line.

What else would you like?

I'll make it freely available.

Darragh _______________________________________________ Blind-sysadmins mailing list Blind-sysadmins@lists.hodgsonfamily.org https://lists.hodgsonfamily.org/listinfo/blind-sysadmins

_______________________________________________ Blind-sysadmins mailing list Blind-sysadmins@lists.hodgsonfamily.org https://lists.hodgsonfamily.org/listinfo/blind-sysadmins

_______________________________________________ Blind-sysadmins mailing list Blind-sysadmins@lists.hodgsonfamily.org https://lists.hodgsonfamily.org/listinfo/blind-sysadmins _______________________________________________ Blind-sysadmins mailing list Blind-sysadmins@lists.hodgsonfamily.org https://lists.hodgsonfamily.org/listinfo/blind-sysadmins

_______________________________________________ Blind-sysadmins mailing list Blind-sysadmins@lists.hodgsonfamily.org https://lists.hodgsonfamily.org/listinfo/blind-sysadmins

_______________________________________________ Blind-sysadmins mailing list Blind-sysadmins@lists.hodgsonfamily.org https://lists.hodgsonfamily.org/listinfo/blind-sysadmins