NetHorus Blog

I’m ready to tear my hair out. No wonder nobody else codes MIB parsers!

Still working. I may come out of this alive…

It’s a frustrating business, this “covert MIB files to something readable with Ruby”. Net-SNMP has it sorted with a %MIB object in perl with information on all the MIB files that it has processed.

The biggest problem I face is that I need to parse all the MIB files using smidump (which I’ve already done), and store them in memory in some half-sensible format which I can then sanity-check and insert in to a database. The sanity checking is what’s taking time.

No wonder so many people do the ‘easy’ bit of SNMP

Crikey. Delving in to SMIv2 has opened up many avenues for problems. Still, I am battling through them and I’m nearly there with a MIB compiler. That’ll get SNMP table retrieval working again, and let me get on with coding an SNMP trap receiver.

I’m not content with hard-coding a load of MIB-specific data in NetHorus. With the aid of smidump and REXML, I am working on converting MIBs to XML and importing them in to NetHorus.

Unfortunately, this takes time and my social life is hectic at the moment. I miss my hour-long commutes and coding time

A month or two ago, I’d noticed some RSpec tests were taking a long time to run. When my ADSL connection dropped out during the day, I found the tests took an increasingly long time to run. Strange – how can lack of Internet connectivity cause some straightforward specs to… aah, it’s because they’re doing DNS lookups all the time.

Brian Candler, a colleague of mine, then published the API-compatible (UPDATE: Brian reminds me it is mostly API-compatible – see his comment) ruby-ip. To quote verbatim:

Why not IPAddr?

Ruby bundles an IPAddr class (ipaddr.rb). However there are a number of serious problems with this library.

1. Given an IP address with a netmask or prefix (e.g. 192.0.2.0/24) it’s very hard to get access to the netmask part. It involves digging around instance variables.
2. It’s impossible to deal with an off-base address with prefix, because IPAddr forcibly masks it to the base. e.g. 192.0.2.53/24 is stored as 192.0.2.0/24
3. IPAddr uses calls to the socket library to validate IP addresses, and this can trigger spurious DNS lookups when given an invalid IP address. ruby-ip does not depend on the socket library at all, unless you require ‘ip/socket’ to have access to the Socket::AF_INET and Socket::AF_INET6 constants.

Today’s checkin waves goodbye to IPAddr, and gives a warm handshake to IP. Thanks, Brian!

I think it’s high time I removed the SNMP proxy.  It’s coded in perl, which is a language I love and have used for years, but not one which fits in to NetHorus in any way that could be considered pretty.

The history is concise – since NetHorus depends on SNMP heavily, it made sense to use Perl’s SNMP library to get things up and running.  I coded a proxy using XMLRPC, which Perl and Ruby both speak, with the intention of removing the Perl proxy at some time.

That time is now.  The SNMP support in Ruby is there, some quick tests have shown it can import MIB files, so here goes…

Three or four months of development is now checked in to Subversion.  Over 90% of the code should be covered by RSpec tests, and there are two important new features:

• Background queue daemon, which queues jobs submitted from the web interface – and later, scheduled and reactive jobs.
• Worker process, which pulls jobs from the queue daemon and does the actual work.  A lot of the code from the Device model is now in a separate library.

And what’s next on the roadmap?

• Built-in SNMP support – no more external SNMP proxy.  I am biting the bullet and using Ruby’s SNMP support… wish me luck as it doesn’t appear to be as good as Net-SNMP!
• Scheduled jobs, such as backing up configurations every night (or more often)
• Cucumber support for testing.  Brian Candler (of deploy2.net) has shown me exactly how cool it is.

I am hoping to get a beta (or at least alpha) release out in the next few months.  That’s not bad going for somebody who has a full-time day job and a pretty busy social life

The cause of the silence this time?  Distributed Ruby (DRb).  It’s pretty magical, and I’m splitting out much of the Rails code in to Ruby code to be executed by a worker process.

Anything long-running, such as inventorying a device, will happen in the background.  Anything instant will still happen in the background, but NetHorus will politely wait.

This is a fairly major change to NetHorus, and it does delay it being usable for a while longer.  Then again, if you need anything usable right now, you shouldn’t be using NetHorus anyway

I am not the best at designing layouts – I’ll freely admit that.  Corinne Pritchard of Simply Understand, on the other hand, is really quite good at it.

With her help (and a giant pizza), we have redesigned the stylesheet and overall layout.  The changes aren’t in NetHorus’ Subversion repository just yet – give it a couple of days.

More changes and updates – it’s been a whole 47 days since the last check-in, mostly due to my tardiness.

Go check it out