Page 1 of 1

Uptime Monitoring Site

Posted: 02/25/09, 10:10 pm
by Dark Shadow
I would like to officially announce the first month of a 9 month development project to build a server and services monitoring software. Here are the features:
1) Remotely monitor services such as HTTP, HTTPS, POP3, IMAP, SMTP, FTP, SSH, MySQL, DNS, ping response and custom ports.

2) Match responses and headers including site content to insure that not only is your site online, but it is still looking the way you left it and not defaced. In addition, FTP check for file and read content to pattern match (including date and words such as success and failure) allows for your backup or scheduled applications that don't normally have notification powers to be integrated into your monitoring software.

3) Verbose error details to assist you in fixing any issue.

4) Online statistics provided on a daily, weekly, monthly, and yearly basis and drawn using pie charts and downloadable in PDF format.

5) Schedule blackouts so that the system does not consider your site offline during regular maintainance periods.

6) 4 monitoring locations prevent accidental reports of downtime. You can choose a single location to monitor your services or choose all 4 to monitor on staggered intervals to insure that your site is actually down not just impossible to route to from certain IP addresses. You can choose if the system warns you after the first failure or from compounded failures.

7) Receive alerts via email and text message even internationally.

8) Is your server online but slow to respond? Yes, you can get notified about that too.

9) API access. Integrate the reporting and account access into your own site.

10) Customizeable 2, 5, 10, 15, 20, 25, 30, 45, and 60 minute monitoring intervals.

11) User and Admin sections for setting up your monitoring. Allow users to view reports and admins to setup and manage monitored services.

Aspects of this software will be free, however, due to the strain of providing 4 servers dedicated to this project, low monitoring intervals such as 2 and 5 minute monitoring and SMS text messaging will probably cost to purchase. I expect to be done with this project around November.

Posted: 02/25/09, 11:04 pm
by RedRage
wow, i have one written in C somewhere that does monitoring from the CLI. Was basically written to replace WhatsUp? from IPSwitch, but i coudln't get it to dial a pager then i got bored. It did reporting in html, but with a bit of php work could be quickly converted to PDF.

Don't understand why you would need 4 servers though?

Posted: 02/26/09, 3:39 am
by Dark Shadow
4 servers provide a redundancy for the monitoring system in order to prevent false positives from popping up if a network between X monitoring server and Y monitored server has an intermittent failure.

Nothing is more annoying than getting false positives when your server isn't offline.

Posted: 02/28/09, 8:23 pm
by Gerry
Wow ambitious! Good luck mate, sounds like a sweet project. Don't forget to add support for monitoring of secure sites. Most monitoring software I've seen falls down when it comes to that.

Posted: 03/20/09, 7:29 am
by Dark Shadow
Just an update:

I have some proof of concept code already complete for this project but I still haven't performed a security check on the code so I won't be posting it here until I'm sure I've hardened it. I believe now I will have the project mostly complete within a month.

SSL cert feature (to monitor SSL status) will probably take longer simply due to the complexity of the feature itself. Personally though, I think the neatest feature is to fetch log files via SFTP or FTP and parse them for error or success codes (and time stamps).

Anyone know of some neat ways to generate pie charts? I know there are lots of ways but if you have any good feedback, I would love to hear it.

Posted: 03/21/09, 11:55 pm
by Gerry
Dark Shadow wrote:Anyone know of some neat ways to generate pie charts? I know there are lots of ways but if you have any good feedback, I would love to hear it.

Although Google's default API isn't that programmer friendly so you may want to use something like this:

Will probably require some mods though as I seem to remember it being kinda buggy.

Posted: 03/23/09, 7:24 am
by Dark Shadow
Log File Pattern Matching Engine P.O.C. code: ... _match.php

It is limited to 10MB and has the following features...

1) HTTP, HTTPS, FTP, FTPS, SFTP and password protected versions of all of them.
2) Support for unlimited file sizes and files greater than the server memory (assuming I allowed files larger than 10MB). I've tested it with files 2GB in size.
3) Throttling control to prevent over usage of system CPU.
4) Selectable case (in)sensitive pattern matching.
5) And yes, it works for web pages as well as log files.

Can you guys think of any other useful protocols to fetch files over besides TFTP? I am also not really interested in building support for SNMP or IMAP/POP (IMAP/POP/SMTP monitoring will be a part of another system) but I'm thinking about building WebDAV support.

Posted: 03/24/09, 2:15 am
by Dark Shadow
Well, WebDAV is out. That's a bit too complex for what I'm trying to do I guess and I don't know any backup systems that would be able to post a file to a WebDAV system better than an SFTP system, so... yeah.

The final version of this will be simply an API called from a cronjob when the event is due. It will support the following:

Good Values (Things that are required to be in the log)
Bad Values (Things that shouldn't be in the log)
Date Value (Date code defineable by the end user. Date code inside the log must be within 24 hours of the cron run time or error.).
Variable Log File Name (Some log files are date-coded. Log file name can contain date variables such as month number 1-2 digit, month number 2 digit, month short name, month long name, day 2 digit, day 1-2 digit, day short name, day long name, 4 digit year, 2 digit year, or any combination of the above. It would look like this when finished. Filename-[dtd]_[mtd]_[yfd].log).