NetHorus Development Blog

It appears that removing ActiveScaffold is a longer task than I thought.

Right now, and it’s not in SVN HEAD yet, there’s a more-or-less working (but it needs prettying up) framework for administering devices.  I hope to check this in over the weekend, or perhaps on Monday, and then get back to work with the BRIDGE-MIB.

Progress of late has become sluggish.  The Rails 2.1/ActiveScaffold 1.1.1 problems have resulted in fighting fires and trying to find magic version combinations that work.

On the train home this evening, I had a revelation.  “Why am I using ActiveScaffold?  I don’t need the flashy AJAX functionality”.

So, ActiveScaffold is disappearing from NetHorus.  This isn’t an “ActiveScaffold is bad”, it’s a “It doesn’t do what I need it to do”. I’d rather invest time in writing my own straightfoward front-end, than spend the same amount of time tailoring ActiveScaffold.

So what about the bridges?  I’m very pleased to report that despite the BRIDGE-MIB being very scary indeed, things aren’t too bad.  I won’t be checking in code to Subversion until such time as the bridge management functionality can show something good, like drawing a basic STP topology.

Finally - it seems nobody out there reads this blog.  If you do, can you get in touch?

Has it really been a week!?

• Presentation - the device management page is now tabbed and takes up less screen ‘real estate’, and it’s more fun to use.
• ActiveScaffold - now works on the current version of Rails in Debian ’sid’.  It was broken for a couple of weeks.
• Visualisation - thanks to GraphViz, it’s now possible to draw hierarchical diagrams of interfaces and hardware.  You will need to install the GraphViz gem.
• Minor fixes - plenty, in particular searching for a device by serial number now works.

I’m really happy with the way NetHorus is progressing.  Check out the timeline on Trac for the latest checkins.

Being full of hayfever and lacking decent air-conditioning in the office has slowed down development temporarily, but at last - there’s an end to the 40s delay in inventorying a device!

The two other notable points:

• Searching for an interface based on Layer 2 address is now supported.
• The interface detail page shows parent and child interfaces.

Over the coming week or two, I hope to finish support for the physical entity MIB and come up with a way of showing relationships between interfaces and cards.

It’s been just over a week since the last update, and over a week since the one before that.  I want to reassure you that development is still continuing, hampered somewhat by the mysterious failing of ActiveScaffold on my Debian development systems.  I still have no idea why, and troubleshooting is proving difficult given the limited time I have right now with work and stuff.

Anyway, the bits you really want to hear:

• Unit testing - the SNMP proxy now has a couple of unit tests
• Troubleshooting - hampered slightly by a bad security fix in Debian’s Net-SNMP package and mysterious RPC-XML proxy delays, there are now benchmarks in the Device class to track down slow SNMP responders, and lots more debugging information in the SNMP proxy
• Little bugs - the adslLineTable is only displayed for interfaces whose type is ‘adsl’ and where there is no physical connector present; interfaces that no longer exist are deleted from the database; the interface traffic monitor has gone, and the ent_physical_element table has been renamed to fall in line with Rails’ naming conventions, we now log when ifHighSpeed is used for a more accurate reflection of interface speed, enums are returned in numeric and text format
• NEW - We now use EtherLike-MIB objects (currently to display interface duplex)
• NEW - Support for the Entity Physical Table, which will be used with the Entity Logical Table to build up a hierarchy of a device’s build and allow ‘impact analysis’ - showing what will happen if a card fails.

Despite a public holiday on Monday, there hasn’t been that much visible progress.  Fear not, most of the stuff is waiting a Subversion check-in, because it’s not working properly yet.

Here’s the round-up:

• Database stuff - migration 039 didn’t work on sqlite, which has been fixed, and Layer 2 addresses are now stored in upper-case by default.
• Network matching - it is now possible to use the ipv4_network? method on a L3Interface to return the Ipv4Network which has the closest match to the IPv4 address.
• Entity MIB support - the ENTITY-MIB now appears in the mibs/ subdirectory, along with an EntPhysicalElement model and a migration.
• Testing - a rather major, but subtle, mix-up between = and == has been fixed.  Some tests failed, but they all run cleanly against MySQL and sqlite now.  Fixtures are now being used for testing, thanks to help from Brian Candler.

That’s it.  Keep watching and suggesting features to me - peter.hicks@nethorus.org.

What a miserable week it’s been for the weather! Here are the latest developments on NetHorus this week:

• Error messages - in particular, the flash messages, are now much prettier and will grey the background out to make the user aware something’s gone wrong.
• Configuration files - NetHorus will now check that a configuration file has been loaded, and raise an error if it hasn’t.
• Testing - we have some tests for the EventLog model, updated tests for the Ipv4Network models.
• Interface dependencies using the IF-MIB::ifStackTable are now working, although the table isn’t straightforward to parse. Credit to Brian Candler for explaining the acts_as_tree plugin to me.
• Model changes - the Ipv4Network model has been tidied up - testing proved that there were some cases where invalid data would be passed and cause errors.
• Credits - Martin Gleadow is now credited properly for keeping tabs on dependencies for the latest releases - something I should write up in the Wiki soon.
• Bugfixes - we destroy_all event logs for a device, rather than delete_all-ing them. This removes the data from the table, rather than breaking the device-to-eventlog relationship and leaving the data sitting stale.
• Interface layout - improvements to make things look prettier and simpler.
• Device inventorying now uses IF-MIB::ifHighSpeed if present to record the speed of an interface, since it appears generally more accurate than IF-MIB::ifSpeed.

So, that’s the past week. How did this week’s planned features go? Not too well…

• SNMPv3 support is still incomplete, I will be de-prioritizing this as it isn’t strictly required for any new functionality, but will be a very-nice-to-have.
• VRF support is lacking - the VRF Lite functionality on my Cisco 877W is slightly limited. I’m aware that full-fat VRF on bigger Cisco boxes works well, and I’m aiming to get started with VRF support this week.
• ifStackTable support is complete in the back-end database, however I want to come up with a way of displaying interface relationships visually.

And finally, here’s what I have planned for the coming weeks:

• ARP table polling - read in the ARP table from a device periodically and use it to track the Layer 2 addresses of devices on the network, and will also be useful for device discovery (incidentally, something I despite but I’m aware it’s a good way to get up and running).
• Layer 2 Forwarding table polling - used in combination with ARP table polling, will allow us to track devices throughout the network.
• Cisco CDP support - for displaying connected neighbour switches.
• ATM VC/VP information - for displaying details of virtual circuits and virtual paths on an ATM switch.
• Frame Relay PVC information - to display PVCs on an interface, their details, etc.
• Neighbour relationships - automatically detecting which devices share common subnets.
• Configuration generation - for Nagios primarily, so we can generate configurations based on the topology information stored in NetHorus. No need to set everything up once in NetHorus and again in Nagios!

Phew. That’s a lot of stuff, but it gives you an idea of what I have planned for the future.

As always, your feedback is important - please email me: peter.hicks@nethorus.org.

Summer is finally here - the last few days in the UK have been pleasantly warm. Us Brits are much happier when there’s direct sunlight!

Despite first glances, there’s been a lot of work going on this week - here’s a summary:

• Error handling and testing has been this week’s theme. If the SNMP proxy isn’t working, or we receive errors, there’s a much prettier ‘flash’ that appears on-screen.
• SNMP improvements - in particular, we now supply MIBs for use with NetHorus as there are some really rather badly maintained MIBs out there. These hang out in mibs/ and will automatically be loaded on startup.
• Application configuration is taken care of by appconfig.yml. You will need to rename config/appconfig.example to config/appconfig.yml for the application to work properly!
• SNMP proxy support has been greatly improved, thanks to Brian Candler’s suggestions. The proxy will now daemonize, write a PID file, provide debugging information… much more useful than before.

Features in the pipeline - this week, if I get time amidst reunions with old colleagues and trips to Dublin:

• SNMPv3 support as suggested by Martin Gleadow. Some of the database schema is already in place.
• VRF support and the ability to add VRF information in to interface inventories.
• ifStackTable support so we can identify which interfaces are children of which other interfaces - this will become useful later on in diagramming and monitoring

Please continue to send in your feedback to peter.hicks@nethorus.org - and note I’ve fixed email now so it won’t bounce. Oops.

Another week has flown by!

• Code tidyups and lots of them - more comments and more error trapping, some tidier HTML and tidier configuration files.
• Device inventorying has been improved and we now take notice of the ifConnectorPresent OID for each interface so we can identify physical and logical interfaces.
• Device-specific information is now in place for ADSL interfaces. If you have an ADSL interface, try examining the interface detail for the interface ending -adsl.
• SNMP proxy now loads all MIB files.

I gave a very quick and hairy demo of NetHorus to a couple of people I met on Thursday, and I’ve had some vague interest from a few other people.

Please spread the word and mail me with helpful suggestions on what you want to see implemented.

It’s just over a week since the last update, and thanks to being away from work for a few days, I’ve had plenty of time to hack away at Ruby.

• Milestone - the 100th check-in!
• Unit tests for some of the models, a process which is remarkably straightforward and I’m really surprised I didn’t grok how easy this was before now.
• SNMP proxy functionality is now working - this requires perl, SNMP.pm and a collection of MIB files. Remember to start external/snmp_proxy.pl!
• Database changes - the iftype association is gone, since this was a hack to get around Ruby’s lack of MIB-parsing.
• Model changes - device models now have get_oid and get_table methods which use the SNMP proxy, providing a useful abstraction from the guts of the SMP proxy.
• UI changes - the interface layout has been refreshed.