9 

 

8   Development

8.1  Known Issues

There are a few known issues that are being addressed:

Please visit bugzilla.

8.2  Database (DBMS) Notes

NetReg was designed using the MySQL DBMS. Why did we choose this platform?

  1. MySQL is relatively easy to build, install, and manage. We are primarily networking people, and don't want or need a fulltime database manager. MySQL can be easily managed as part of many other pieces we support.
  2. MySQL is open source, and we like open source software.
  3. MySQL offered the features we feel are critical.

Some criticism is levelled against MySQL for not supporting foreign keys or transactions. Recent versions of MySQL use the InnoDB table type to support both of these methods. We have our database using InnoDB, as they are more resilient to corruption in our experience. We have not yet made use of foreign keys or transactions in NetReg yet, though NetMon is using transactions in some functions.

Now, if you want to port NetReg to use another DBMS, here are some of our thoughts:

  • It should be relatively easy to use a different Perl DBD module. Just look in CMU/WebInt/config.pm and change the appropriate variable.
  • All in all, we believe most of the DB features used are fairly standard, but an exhaustive comparison to other DBMSs has not been done. However, knowing what we do, the following structures that NetReg uses may not be widely compatible:
    • We use statements such as
      SELECT COUNT(DISTINCT())
      Ironically this was supported in MySQL "Just in Time" for NetReg
    • The schema uses types such as TINYINT, SMALLINT, INT, and TEXT, which may be incompatible

8.3  Change Log

8.3.1 Stable Release 1.1.7

Release 1.1.7 contains some under-the-hood improvements and bug fixes.

Noteworthy changes in 1.1.7 are:

Cascade Deletion

A new cascade deletion / update engine enables a high degree of data consistency in an environment without foreign keys (MySQL pre-InnoDB). Even with DBMS foreign keys, the system can enforce similar constraints at the places in our schema that can reference one of several tables.

The benefit of this system will be greater consistency, as deletion of dependent elements will either not be permitted or will be done automatically with the deletion of a master element (for example, DNS resources of a zone being deleted will be cleaned up automatically).

8.3.2 Stable Release 1.1.6

Release 1.1.6 contains new features and bug fixes. If you are running release 1.1.4 or 1.1.5, you should read these notes and use the support/convert/convert-1_1_5-1_1_6.pl script, and test your upgrade plans on a non-production server prior to rolling it out.

The major changes to 1.1.6 are:

Installation Improvement

The netreg-install.pl script has been improved; it now has the ability to read Netdb/config.pm and WebInt/config.pm and configure the variables in each. Certain variables are flagged as important, and netreg-install can present only the important variables, or all the variables, for configuration.

Improved Netdb Debugging

Throughout the Netdb modules, all error debugging is tagged with the file and line number generating the output.

Protection Profiles

Adding new subnets, zones, and services results in automatic ADD access granted to netreg:admins. This is implemented by "protection profiles", controlled via the Netdb/config.pm module.

DHCP Subnet Default Options

With this release, all of the options in the DHCP configuration file can be controlled from within the web interface/database. Additionally, so-called "subnet default" options can be specified. Assuming no subnet-specific option, a subnet default option will be added to the configuration.

Other features/bug fixes

Minor features and bugfixes include:

  • QuickReg bugfixes - eval { } protection of omshell
  • Error messages reported to user can now be recorded in the Apache logfile
  • Improved basic error messages
  • CNAMEs may now be added to any zone permissible on the subnet by any user (previously non-admins could only add CNAMEs to the same zone as the primary machine)
  • DHCP failover declarations use IP addresses, not hostnames.

8.3.3 Interim Release 1.1.5

Release 1.1.5 corrects minor fixes in the 1.1.4 release; namely, several files were sticky tagged in the CVS release sandbox that should not have been tagged.

8.3.4 Stable Release 1.1.4

Release 1.1.4 contains new features and bug fixes. If you are running release 1.1.3, you should read these notes and use the support/convert/convert-1_1_3-1_1_4.pl script, and test your upgrade plans on non-production server prior to rolling it out.

The new features of 1.1.4 are:

Secondary IP Support

Support for secondary IP address interfaces of machines has been added. This enables a single MAC address to be registered to more than one IP address on a subnet. However, only the 'static' (primary) registration will be propogated into DHCP. If you have a server providing services on multiple IPs (for example, an SSL-protected web server with multiple virtual domains), this enables the secondary IP addresses to be registered using mode 'secondary'

QuickReg Feature

QuickReg is a new feature that provides the fastest possible registrations to users. NetReg queries the DHCP server for the MAC address of an unregistered host, and determines the correct registration subnet. It's "one-click" registration.

"Hot Keys" in Web Interface

Using new HTML features, we have enable "hot key" support throughout NetReg. In NetReg forms, pressing Alt-, where is the first letter of the field name, will toggle the cursor to this field.

Bulk IP Registration Fix

Bulk IP Registration (a link on the Subnet Information page) was introduced in prior versions. However, there were fundamental bugs involving the user and department information attached to the new registrations. The web front-end provides fields to enter this information.

Removed DNS Server Software Section

Prior to having the DNS server configuration information derived from service groups, there was a dedicated section to handle these operations. This section has been inaccessible in the past few releases, but this release officially removes all of the relevant code.

Advanced Permissions Matrices

Access to perform certain features (such as change the IP address of a host without being an administrator) has been implemented. See the doc/netreg-permissions.txt file for the exact permissions required to perform certain operations. Not all operations are documented; we are hoping to improve the completeness of this file with time.

8.3.5 Stable Release 1.1.3

Release 1.1.3 contains new features and bug fixes. If you are running release 1.1.2, you should read these notes and use the support/convert/convert-1_1_2-1_1_3.pl script, and test your upgrade plans on a non-production server prior to rolling it out.

The new features of 1.1.3 are:

"Switch User" capability

This feature enables administrators to look at NetReg and see exactly what another user sees. This is very useful in debugging and user education. When acting as another user, an orange bar appears at the top of each web page, indicating the alternate user and providing a link for logging out. This is implemented as an extra cookie. Verification is done on each page load that the true identity is that of a SuperUser (users of this feature must be listed in the SuperUsers array in CMU/WebInt/config.pm). Access to this function is via the "Users/Groups" area of NetReg.

Outlet Subnet Membership/Vlan capabilities

Outlets can now exist on a single subnet or multiple subnets. They can be configured as a trunk port, with appropriate settings. This feature is useful for enabling VoIP (using Cisco's "voice vlan" settings). The backend is done via Perl and Expect (in terms of actually setting vlans, etc.). Long term the back-end will most likely be extracted out completely.

NetMon InnoDB Tables

At this time, we highly recommend the use of MySQL's InnoDB table types for NetMon. We are currently running version 3.23.49a of the server; 3.23.50 promises to solve a few bugs (keeping foreign key references intact across an ALTER TABLE, for one). The InnoDB table offers substantially improved parallelism with row-level locking. Additionally, it provides support for transactions and foreign keys. The code base has changed to use the InnoDB features (such as the SELECT ... FOR UPDATE syntax), thus you will most likely want to stay with 1.1.2 and non-InnoDB tables, or upgrade both.

NetMon Topology Improvements

The NetMon Topology code has undergone substantial improvements in determining the exact layer-three network topology. Unfortunately, the code in 1.1.3 is most likely not going to provide any useful output, as it is still undergoing heavy reorganization. However, if you want to see the latest and greatest, check it out.

Improved Consistency Checker Routines

Several consistency checking routines have been added.

DNS Resource Searches

You can now search the DNS resources table from the web interface. There is a link to this search off the "Search Machines" form. Additionally, traditional machine searches that would also be legal as a DNS resource search (ie, just searching by hostname) will generate a link to an equivalent DNS resource search if no machine results are found.

DNS Load Balancer Available

Release 1.1.3 features our newest DNS load balancer. This load balancer uses DNS UPDATE to perform updates against specific zone records based on input from the members of a pool. The method of collecting this data (such as number of users, number of processes, etc.) is modular. We have written collectors that use SNMP (standard Unix MIBs) and our in-house "KMemSrv" server. Updates are performed very rapidly and react to the gathered statistics.

Permissions Updates

The file doc/netreg-permissions.txt now contains the beginnings of documentation about the exact permissions required for various operations.

8.3.6 Stable Release 1.1.2

Release 1.1.2 contains a number of changes and new features. If you are running release 1.1.0 or 1.1.1, you should make sure to read these notes, use the support/convert/convert-1_1_1-1_1_2.pl script, and test your upgrade plans on a non-production server prior to rolling it out.

The new features of 1.1.2 are:

Service Group Membership

Service Groups previously were composed of member machines and attributes. We have extended the service_membership table and related functions to support membership of many other NetReg structures, including subnets, zones, users, groups, and even other service groups. This provides amazing flexibility in defining additional service types.

DNS Server Configuration

Previously DNS Server configuration (zones, master/slaves setup, etc.) was controlled through the DNS Server area. This has been changed to be controlled as a number of service groups. While this is slightly less intuitive, the additional flexibility outweighs this drawback. You can easily define additional attributes of DNS servers, and then just update the dns.pl output script to translate your attribute into configuration bits -- no work to the Netdb or WebInt trees are necessary.

Be sure to re-read the DNS server configuration sections for a full description of the current service group implementation.

DHCP Option Validation, DHCP Classes

We have written a full recursive-decent parser to validate the format of all DHCP options. This should make it safe to allow end users to define DHCP options for machines under their ownership. In previous versions this was not recommended because an invalid option specification would break the DHCP server configuration.

The syntax for the DHCP Option format closely mirrors that of the ISC DHCP server format (man dhcp-options.) It is fully described above.

Additionally, we have added a DHCP Class service group for definining various DHCP classes. The only attribute is the match parameter string. However, service groups were extended to allow DHCP Options to be attached to them. The options attached to a DHCP Class service group are included in the class definition in your DHCP server configuration.

Finally, all DHCP Option Type (specifications) now include a builtin parameter for specifying whether a given option is known by the DHCP server, or not. Options that are not 'builtin' will have the output format specification added to the configuration.

Scheduler Frontend

We added a web interface to control the scheduler's tasks. User/groups with Level 1 WRITE access will be allowed to run the 'Force Now' or 'Back Off' operations to control scheduling of certain tasks. Users with Level >= 5 WRITE access will be able to perform these operations, as well as change the name, default interval, or blocked time of a task. The blocked time is a hard "next run" time. The point of the interface was to allow a wide(r) group of people have some control in critical periods of DNS/DHCP propogation, while the smaller administrator can set a blocked time if work is being done, etc

8.3.7 Interim Release 1.1.1

This release changed a few default configuration options for the use of the general public.

8.3.8 Initial Public Release: 1.1.0

This is the initial public release. The entire manual has been written for this purpose, as well as the development of initial database data. We welcome all comments, suggestions, offers to fix things, etc. There have been a number of changes since the initial release; too numerous to list.

8.3.9 Initial Release: 1.0.0

The initial release of August 14, 2000, was the first release to campus users of NetReg.

8.4  To-Do

8.4.1 In Progress

The following changes are in-progress:

  • User Attributes: Change "service attributes" to also include user/group attributes (schema changes already done). Allow user attributes and develop user preferences, to include persistent sort order of machines/outlets, etc. Changes in progress, no target date.
  • Stop using the BIND 'nsupdate' utility for performing DNS updates; instead use the Perl Net::DNS::Update package.

8.4.2 In the Future

The following changes are desired, but no active development is currently occurring:

  • User attribute: selectable disabling of paging on various screens
  • Make the outlet database more generic (not as specific to our labelling standard, etc.)
  • Implement CSS throughout NetReg; pull CSS from WebInt/interface.pm and put it into a separate styles.css file
  • Make the subnet map interface more generic (not pegged to 128.2 and 128.237 networks): base the list on subnets
  • Cleanup dns_resource rname_tid usage -- make it the canonical reference where appropriate

8.5  Carnegie Mellon Developers

If you need help with NetReg, please use one of the contact methods listed earlier (sending mail to netreg-bugs@andrew.cmu.edu is one). This will lead to speedier resolution of your problems.

Kevin Miller

  • Developed most of the main NetReg codebase
  • Designed, developed most of the NetMon system
  • Developed all DNS zonefile/DDNS update operations
  • Overhauled DHCP and other output scripts
  • Wrote this manual
  • Developed attribute/attribute spec and formatting
  • Developed several of the reports (subnet usage, subnet map)
  • Developed DNS Load Balancing system

Dave Nolan

  • Developed NetDB skeleton/ground work
  • Extensive work to Telecom cable management system
  • Developed outlet vlan configuration system
  • Numerous patches for bugs, fixes, etc.

Chaskiel Grundman

  • Wrote the consistency checker

Kunal Trivedi

  • Subnet policy pages
  • LDAP and local user group

Ed Bardsley

  • Cleaned up DNS resource and DNS zone problems
  • Developed secondary IP support
  • Implemented hot keys throughout the web interface

Russell Yount

  • Project Manager
  • Initial database schema design

Ros Neplokh

  • Initial DHCP configuration generation
  • Initial DNS configuration file generation

Jim Foraker

  • Designed initial outlet interface

Ken Tew

  • Developed additional reports