After installing Fedora 16 on it, I was a little disappointed in how its video worked with my HP 25″ display. It was kind of slow and I noticed the mouse could move off the left edge near the top. Huh? There is only one video output from this motherboard, but it was acting like there were dual displays. A quick review of xrandr revealed an output LVDS1 which is for a laptop panel and does not exist of course.

So that led me to review the i915 kernel module parameters using modinfo. It suggested to me I could instruct the module to ignore the laptop panel display and also enforce modesetting on the VGA port only. I attempted this by editing /etc/grub2.cfg and appending these two kernel options:

i915.modeset=-1 i915.panel_ignore_lid=-1

After a re-boot, I was instantly gratified by the super-fast boot (5-seconds for Linux and another 10-seconds for automatic login to GnomeShell to complete). For $69, the speed and versatility of this motherboard/CPU is outstanding. I will have to contemplate ordering more, heh.

It has been widely popularized by Boston fans that without Belichick, Parcells would not be known as a head coach football icon… hopefully, that is from more of a knock against the NY media than the accomplishments by the man.  But I am just as certain that NY fans say the converse against the Boston media, too.

Personally, I think Crennel (and Weis) were the real enablers of that media-hyped dynamic duo, because where would both of them be without Crennel?  I throw (and Weis) in there, because it’s not his fault he was missing prior to the 1990 campaign, but he certainly has a strong NFL resume when Weis is joined with Crennel.

Here are my points to support this claim:

1)  Playoff record of Parcells and Belichick, with Crennel (and Weis):

… as World Champions:

1986: Parcells, Belichick, Crennel (NYG, 17-2)
1990: Parcells, Belichick, Crennel, Weis (NYG, 16-3)
2001: Belichick, Crennel, Weis (NE, 14-5)
2003: Belichick, Crennel, Weis (NE, 17-2)
2004: Belichick, Crennel, Weis (NE, 17-2)

… reached the Super Bowl:

1996: Parcells, Belichick, Crennel, Weis (NE, 13-6)

… reached the Playoffs:

1984: Parcells, Belichick, Crennel (NYG, 10-8)
1985: Parcells, Belichick, Crennel (NYG, 11-7)
1998: Parcells, Belichick, Crennel, Weis (NYJ, 13-5)

The duo is 20-4 in the playoffs and 5 rings, because they had Crennel (and Weis); with Weis, 15-2 and 4 rings.

Now, the above alone does not make my point complete, unless you combine it with Belichick and Parcells resumes, which are replete with mediocrity:

2)  Parcells and Belichick as head coaches, but without Crennel (and Weis):

1983: Parcells (NYG, 3-12-1), inherited 17-24 (4-5, 9-7, 4-12) team, 1-1 playoffs

1991: Belichick (CLE, 3-13), inherited 22-25 (3-13, 9-6, 10-6) team, 1-2 playoffs

1992: Belichick (CLE, 7-9)

1993: Belichick (CLE, 7-9); and Parcells (NE, 5-11), inherited 9-39 (2-14, 6-10, 1-15) team, no playoffs

1994: Belichick (CLE, 11-5), 1-1 playoffs; and Parcells (NE, 10-6), 0-1 playoffs

1995: Belichick (CLE, 5-11); and Parcells (NE, 6-10)

1999: Belichick (NE, 8-8), inherited 30-18 (9-7, 10-6, 11-5) team, 3-3 playoffs

2000: Belichick (NE, 5-11); hires Weis

2003: Parcells (DAL, 10-6), 0-1 playoffs, inherited 15-33 (5-11, 5-11, 5-11) team, no playoffs

2004: Parcells (DAL, 6-10)

2005: Belichick (NE, 10-6), 1-1 playoffs; and Parcells (DAL, 9-7)

2006: Belichick (NE, 12-4), 2-1 playoffs; and Parcells (DAL, 9-7), 0-1 playoffs

2007: Belichick (NE, 16-*), 2-1 playoffs

2008: Belichick (NE, 10-6)

2009: Belichick (NE, 10-6), 0-1 playoffs

2010: Belichick (NE, 14-2), 0-1 playoffs

2011: Belichick (NE, 5-3), playoffs?

… Playoff record without each other, AND WITHOUT CRENNEL (and WEIS):

Belichick (5-5), 0 rings
Parcells (0-3), 0 rings

It can be noted here that Belichick alone has had a lot more opportunity, but he also inherited successful teams, whereas Parcells inherited dismal teams.  But those teams did not become legendary until Crennel (and Weis) joined them.

The result is this Tcl/Expect script (listed below).  First, you will need to make certain you have these dependent packages installed:

yum install expect fluid-soundfont-lite-patches timidity++

Next, you will need to fetch the multimedia files into your home directory:

svn co https://robert.hurst-ri.us/svn/rpgd/trunk/snd Music/hack

Then, setup your telnet session attributes in .telnetrc to automatically load when you connect:

robert.hurst-ri.us
    mode character
    set crlf off
    unset escape

And last, simply copy/paste the script below into an executable text file of your choice and run it.

#!/usr/bin/expect -f

# play a MIDI file
proc midi s {
  set mid [string range $s 3 end-3]
  set bool [string range $s end-1 end-1]
  set cmd "killall playmus 2> /dev/null; playmus -v 25 Music/hack/$mid.mid &> /dev/null &"
  system $cmd
}

# play a WAVE file
proc wave s {
  set wav [string range $s 3 end-3]
  set bool [string range $s end-1 end-1]
  set cmd "play Music/hack/$wav.wav &> /dev/null"
  if {$bool == 0} { set cmd "$cmd &" }
  system $cmd
}

# connect to the game server ...
log_user 0
spawn telnet robert.hurst-ri.us 7000
wave "...dungeon;0."

expect_background {
  -re "\033\\\[\\\{.*\\\}" { midi $expect_out(0,string); return }
  -re "\033\\\[\\\[.*\\\]" { wave $expect_out(0,string); return }
  -re "(.*)" { send_user -- $expect_out(0,string); }
}

interact

Over the years, its onboard fan got a bit louder and louder, which I believe was a result of maturing Linux graphic drivers that were pushing its dual GPUs harder and harder.  So much so, I could hear my workstation whining away from another room.  That’s not good.  No matter what TLC I put into the workstation to improve firmware, air flow, etc., I could not get rid of the annoying decibels it was throwing about.

This card performs admirably, supporting both my HP 24″ displays at 1920×1200 each, but it runs super quiet.  I cannot hear any part of the workstation’s fans when exiting the room — and it has 7 of them: three upfront, one rear, one on each CPU, and the one on this new video card.  I can sit comfortably now at my workstation and hear myself think again — it’s like meeting myself new all over again, and that’s not necessarily a bad thing either.

I can only imagine our dog, Rebecca, must be feeling without all that air swooshing in that background. 

Hurst, Kenneth
posted September 6, 2010

Kenneth Hurst, 87, of Chattanooga, died on Sunday, Sept. 5, 2010.

He was the son of Queenie Saxson and Alonzo Hurst, of Providence, R.I. His siblings are Elenor Fenley, John, Frances Magnum, and Ronald all of Providence. He was a graduate of Mount Pleasant High School, Rhode Island Technical School, and Rhode Island School of Design.

Kenneth was Providence Junior Horseshoe Champ at age 11 and New England Champion in 1939. Kenneth was also a state Soap Box Derby Champion.

During World War II he was a Technical Sergeant and served as an aerial gunner/flight engineer on a B-24, flying 34 missions with the Eighth Air Force, over Germany. He was awarded the Distinguished Flying Cross, Air Medal with three oak leaf clusters, EAME Campaign Medal with four battle stars, and the Good Conduct Medal.

Kenneth was married to the late Muriel Lusk of Altamont, Tn., for 63 years.

His children are Michael, of Birmingham, Karen (Batson), of Wilmington, Steven and Vincent, of East Ridge and Hollye, of Asheville. Kenneth had ten grandchildren and four great-grandchildren.

He was employed with Combustion Engineering (Alston Power) for 20 years. He was founder and majority partner of Metallurgical Enterprises in Lakeview.

He was a member of Jones Memorial United Methodist Church.

Visitation will be from 12-1 p.m. on Wednesday, Sept. 8, at the funeral home, with funeral services at 1 p.m.

Burial will follow at Chattanooga National Cemetery.

Arrangements are by the East Chapel of Chattanooga Funeral Home, Crematory, and Florist, 404 South Moore Road.

I hardly knew him, but more through story and reputation. What a life.

It is curious to me to hear from the average fans that Brady had this perception of being “dissed” throughout the Patriots’ winning ways; especially back when he led them (completely) to back-to-back championships (2003, 2004) and took credit for the first one (2001) provided by Drew Bledsoe’s off-the-bench winning performance in his second AFC Championship game while “The Team of Destiny” brought home the improbable win against the heavily-favored, 2000 champion St. Louis Rams. But what did the average fan expect, all the while Genius and Terrific were (selflessly?) promoting “The Team” above “individual stars”? Did you want it _both_ ways? Do you believe that without QB Tom Brady performing at the helm, “The Team” had no chance? If that is so, then what exactly was the Coach-QB tandem selling to the team, the media, and its fans?

But after pedestrian 2005 and 2006 playoff appearances (Terrific was not so with the two losses coming from 3 interceptions thrown, and despite “The Team” and referees bailing his ass out with another improbable win when he delivered 3-interceptions to San Diego’s defense), the 2007 perfect* regular season finally brought Terrific some personal recognition: both the NFL MVP and Offensive Player awards. Sure, he got two Super Bowl MVP awards (with the highly debatable first one), but not too many non-QB players get those game honors to begin with, so not of his cause. No, despite the addition of an elite game-breaker in Randy Moss, who was apparently the lucky recipient of a record-breaking 23 TD tosses from Terrific, the personal glory was squarely Brady’s own. I wonder just how many of the other 52 “professional athletes” were genuinely agreeable and supportive?

I am guessing it fell short of consensus. The wave of media love that has followed Terrific since has swelled proportionately with the NFL Golden Boy’s head since 2007. I think Genius and Terrific are believing the hype, which usually spells doom for those who follow them. Is that justified? After all, Tom Brady’s career regular season wins he is credited for (.776) is untouchable by his peers: #1 “statistical QB” Peyton Manning (.678) and #1 “clutch QB” Ben Roethlisberger (.704).

And yet, this current losing streak of historical importance started after that national personal validation that Terrific _is_ an elite QB. But what followed was a marked dropoff in personal performance, which really began with another improbable win against San Diego in the AFC Championship game. MVP Tom Terrific came up small with a 66.4 rating from another 3-interception game. But that got him another handed opportunity to the 2007 Superbowl against the New York Football Giants, whereas Tom Terrific blew a perfect* season and a chance to sit alongside QB titans #1 Joe Montana and #2 Terry Bradshaw — not only did both lead their teams to all four world championship opportunities, but they also individually performed at a high level recording 101+ passer ratings in each game. Terrific did record an efficient 86.2 rating in his first appearance/win, just perhaps not worthy of the MVP honor endowed; as well as an acceptable 82.5 rating in his loss against the New York Giants championship defense:

Follow up that haunting nightmare of one epic loss with missing the 2008 season with an injury. Wow, Terrific must have worked his butt off in rehab, because he came back in 2009 listed as an inch shorter down to 6’4″ and 13 pounds lighter down to 225. Terrific has to give some credit to the world-class trainers and rehab facilities, because he was listed as a 6’5″ 205-lb rookie and got 33 pounds “bigger” at the 2001 training camp. That is pretty amazing stuff — Tom is a terrific human being.

But “The Team” somehow went on without Terrific and played to a very respectable 11-5 season, and only to miss the playoffs because they were bested by a pivotal NFL Thursday night pay-per-view overtime loss to the legendary QB Brett Favre and his NY Jets. We can only wonder what Terrific might have done in 2008. In the offseason, I suggested “The Team” keep the younger and now game experienced Matt Cassel and trade away Brady to Buffalo. Think of the TV ratings we could have gotten with the “Brady Bowls”? Instead, we kept an older and now gimpy Tom Brady and lost our star GM Scott Pioli, who obviously took notice of Kraft’s reluctance to let go of the past who allowed both Cassel and Vrabel to follow Pioli to the Kansas City Chiefs.

2009 was yet another lackluster regular season, just a one-win dropoff to 10-6, probably because of Terrific’s accelerated comeback from West Coast medicine with under-the-radar rehab. Of course, that got Terrific another gaudy accolade for being a Comback Player of the Year award — it validates his New England branding of being a tough guy (really?). But it was enough to once again go to the postseason AND with home-field advantage where Terrific can convince the referees to throw a flag after an unsuccessful 3rd down. I can almost here Terrific cry, “Do over! Do over!”. D-O-O-M was never spelled more prominently than when the Ravens ripped off an opening 83yd TD run. Oh, wait, that one is on the Genius’ complex defense schemes, not Terrific — instead, the Golden boy proved himself just as good as his predecessor by standing tall in the pocket like a statue, getting stripped of the football inside their own 17 to lead to another Raven’s TD:

But the 2010 Patriot postseason personal stat line looked acceptable enough to win, except that most of Terrific’s passing contributions came late in a game when it clearly did not have any relevance to its outcome. The critical error on their opening drive — overthrowing a screen pass — for a huge interception return was enough to put Terrific and Genius into a spotlight they are not accustomed to and cannot hide from — scrutiny:

Scrutiny for benching Welker on violating Genius’ edicts; scrutiny on Terrific’s personal shrinking act on the big stage: an average 10-point drop in postseason passer rating over the regular season and now three playoff loses in a row, one on a neutral field and the other two on their own turf. WTF??!? Will it really be over 1450-days when the Patriots MIGHT get a chance to win another playoff game?

2011 may yet bring an NFL season to our homes, as they continue to bicker over the unaccounted for billions at their disposal. I don’t see the Brady lawsuit really accomplishing much, since Brees is covering up for Terrific’s absence and doing all of the talking. But if it’s a complete season, then I must chant “go STEELERS!!” If it is shortened, I must rant “comeback Favre!!” and “go BILLS!!” That rant alone should get the talks moving, eh?

Old (VIC20), new (Linux), and virtual computing (Win7) to dual monitors

I got another 24″ monitor (left) and its make is also HP.  It sits comfortably paired with an HP w2408h (right) and its stand also supports tilt and swivel, something HP got right that other makes still don’t get.  It is packaged with a complement of cables: a pair of HDMI, a pair of DVI, and a USB to bridge its ports to the computer.  And of course, it is also 16:10 aspect ratio despite the overflow of cheaper 16:9 monitors on the market.  Do consumers really care about the small black areas above/below a 1080p HD movie playing on their computer display?  I’ll take the 1200 pixel rows over 1080 every time, thank you.

But what attracted me most about this particular model is its multiple video inputs: composite video, s-video, component video, 2x DVI, DisplayPort, and HDMI.  The monitor does a bang up job with the analog inputs, allowing for original and scaling sizes to maintain aspect or fill the screen.  The screenshot shows a VIC 20′s composite video displaying in a Picture-in-Picture display overlaying a portion of the Fedora 15 Linux Gnome Shell background.  This monitor also supports Picture-out-of-Picture (below) which means the widescreen display is split into two halves for side-by-side viewing, a very nice and quite usable option.

Desktop computing is fun again with this much screen real estate (3840×1200) to roam about and easy attachment of a variety of video sources to appease my retrogaming pleasures.  It is truly versatile by every measure.

HP LP2475w POP example

The most obvious change, of course, is the introduction to Gnome 3.0 with Gnome-Shell.  Wow, has it come a long way in usability from my preview of it about a year ago.  After an hour of self-discovery of this shiny new user interface to the Gnome window manager, I am pleased to say I have adapted to it quite readily — and that’s a good indicator.

The UI is intuitive enough to me.  I like the ease of managing its favorites menu and I especially like the window management with the dynamic workspaces to the right.  I do miss some of Compiz features, such as wheel zoom and mouse selecting any area to snapshot to a graphic image, but I am confident such niceties will arrive eventually.  It is a must to install gnome-tweak-tool, simply to increase the font rendering scale and use preferred fonts — as nice as the default liberation fonts look, I have grown accustomed to my preferred combo of Arial, Lucida Console, and Trebuchet.

Firefox 4 is also a welcome change.  It starts up faster and it renders faster — I was starting to warm up to Google Chrome as a replacement, because of Firefox’s prior performance.  I had to first install an extension NoSquint that manages page zooming to leverage my high definition monitor and accommodate my aging eyes.  Next came the usual suspects from Adobe, except they now offer a 64-bit flash player, which means nspluginwrapper is no longer needed; OpenJDK and IcedTea have proven to be capable enough for my Java-enabled web needs; and mozplugger with mplayer and timidity++ (removing totem-mozplugin of course) allows HTML embedded media to play without fuss.  Finally, CUPS with system-config-printer continue to not disappoint with its auto-detection of our networked HP Color LaserJet 2605dn printer.

As a long-time end user of OpenOffice (I still got my boxed copy of Sun’s StarOffice), I am happy to see a parallel fork of it as Liberation Office.  Remember what happened when XFree86 messed with its licensing and became Xorg?  It is a reminder that open source projects like these in danger of getting strangled by commercial interests can quickly become a non-issue when its development community takes charge over those few that would seek to exploit the masses with their shortsightedness goals.

Ok, enough applications and usability talk — time to fire up MAME and other classic emulators and do some retrogaming — woot!

Peoplesoft is setup to do an IPVS direct routing method.  That is, client connections connect into the IPVS director hosting the virtual IP address.  The packet gets redirected by IPVS to a physical host running one of the Weblogic services.  After the client requests are processed by Weblogic, the reply packets get sent back directly to the client machine — without traversing back through the IPVS layer.  This is the most lightweight of methods offered by IPVS.

To configure IPVS for direct routing, all the hosts must be running on the same VLAN.  And since IPVS in this mode does not do any network address translation (nat), all of its virtual service port numbers must match the real service port numbers, i.e., https://hrms.somedomain.com:8443 might redirect as https://blade4.chassis2:8443.

It is important to setup any IPVS timeout that works with your application needs, i.e., our Peoplesoft portal pages have an enforced application timeout of 20-minutes, so IPVS should not drop any idle client connections in advance of that.  Just modify /etc/sysconfig/ipvsadm and add a line like:

--set 1200 0 0

Start the ipvsadm service, which merely echos this file to ipvsadm -R.  You can always change this value on-the-fly, too, so no worries here.  Then configure /etc/sysconfig/ha/lvs.cf to load-balance for two web services on different hosts:

serial_no = 12
primary = 10.25.63.62
service = lvs
backup_active = 1
backup = 10.25.63.63
heartbeat = 1
heartbeat_port = 539
keepalive = 3
deadtime = 6
syncdaemon = 1
network = direct
debug_level = NONE
monitor_links = 0
virtual hrms8443 {
  active = 1
  address = 10.25.63.64 eth3:0
  vip_nmask = 255.255.255.0
  port = 8443
  persistent = 1200
  expect = "200 OK"
  use_regex = 0
  send_program = "/usr/local/sbin/check_hrms_ssl.sh %h 8443"
  load_monitor = none
  scheduler = wlc
  protocol = tcp
  timeout = 6
  reentry = 15
  quiesce_server = 0
  server hrwebp01 {
    address = 10.25.63.60
    active = 1
    weight = 1
  }
  server hrwebp02 {
    address = 10.25.63.61
    active = 1
    weight = 1
  }
}

Start the pulse service and dump the IPVS table:

$ sudo ipvsadm -L -n
IP Virtual Server version 1.2.0 (size=4096)
Prot LocalAddress:Port Scheduler Flags
 -- RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  10.25.63.64:8443 wlc persistent 1200
 -- 10.25.63.60:8443             Route   1      0          0
 -- 10.25.63.61:8443             Route   1      0          0

The last step is to setup each of the real servers running a web service each to process the redirected TCP packets for the virtual IP address, i.e., 10.25.63.64.  So, modify their /etc/sysconfig/iptables and add the following ruleset:

*nat
: PREROUTING ACCEPT [136:11568]
: POSTROUTING ACCEPT [12:1557]
: OUTPUT ACCEPT [12:1557]
-A PREROUTING -d 10.25.63.64 -p tcp -m tcp --dport 8443 -j REDIRECT
COMMIT

Startup the iptables service to enable these rules.  Again, there is no NAT’ing going on here, but these rules ALLOW the redirected packets from the IPVS host to be processed on this host’s networking stack, just as if it held the virtual IP itself.

For a number of years we have enjoyed the high availability successes of IPVS handling this Peoplesoft architecture.  Thus, we have also implemented IPVS for our core clinical information system running InterSystems Caché.  This implementation, however, does not make any use of piranha and does not use the direct routing method.  The main reason why we chose not to use piranha’s pulse / lvsd services is because there is a large mix of stateless (web) and stateful (telnet, ssh) connections into the applications.  And while the pulse / lvsd services could be configured to accommodate these requirements, there are some associated complexities in its configuration, startup, and fail-over scenarios that were undesirable to its system engineers.

Also, we were planning to run multiple Caché application services per physical blade, but alas, those services listen on ALL the host’s adapters (0.0.0.0) and cannot be configured to listen to assigned virtual IP addresses, only port numbering.  The thought of running one instance per KVM virtual guest to circumvent the port conflict was explored, but that introduced an unsupported RHEL 5 cluster configuration (it may be supported using RHEL 6, so that might be a future consideration) since there are integral, shared GFS2 clustered filesystems needed for the application, too.

These limiting factors did not hinder us from moving forward with IPVS.  Instead, we developed our own framework to operate this useful load-balancer without introducing “too much” complexity.  That is, the service configurations are maintained consistently on all physical hosts (dev-test-production), making for simpler startup and shutdown sequences, yet allowing for real-time dynamic changes to any environment without contending with fixed, running daemon services.

The IPVS host must have NAT enabled, so enable that in /etc/sysctl.conf as:

# Controls IP packet forwarding
net.ipv4.ip_forward = 1

The IPVS ruleset can be maintained in /etc/sysconfig/ipvsadm, although we have a cluster service (start, stop, status) script that invokes the appropriate ipvsadm commands, like thus:

start)
  # if either LAN is NOT configured, do nothing but exit normally,
  # because we may be running in some monolithic mode
  if ! alive $VIP ; then
    println "$INSTANCE ($VIP) is not alive"
    exit
  fi
  if ! alive $DIP ; then
    println "$INSTANCE ($DIP) is not alive"
    exit
  fi 

  # ok, setup as an IPVS director
  echo "1" > /proc/sys/net/ipv4/ip_forward
  grep -v '^#' /etc/sysconfig/ipvsadm | $ipvsadm -R
  $ipvsadm -A -t $VIP:22
  $ipvsadm -A -t $VIP:2972 -p
  $ipvsadm -A -t $VIP:9671
  ;;
stop)
  # downtime, teardown an IPVS director
  $ipvsadm -D -t $VIP:22
  $ipvsadm -D -t $VIP:2972
  $ipvsadm -D -t $VIP:9671
  ;;
status)
  # not a running IPVS director, just exit
  [ ! -f /proc/net/ip_vs ] && exit
...
  ;;

The result of this IPVS host looks something like this:

$ sudo ipvsadm -L -n
IP Virtual Server version 1.2.0 (size=4096)
Prot LocalAddress:Port Scheduler Flags
 -- RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  10.25.63.79:22 wlc
 -- 192.168.2.131:22             Masq    1      1          0
 -- 192.168.2.132:22             Masq    1      0          0
 -- 192.168.2.133:22             Masq    1      1          0
 -- 192.168.2.134:22             Masq    1      0          0

TCP  10.25.63.79:9671 wlc
 -- 192.168.2.131:9671           Masq    1      105        0
 -- 192.168.2.132:9671           Masq    1      103        0
 -- 192.168.2.133:9671           Masq    1      103        0
 -- 192.168.2.134:9671           Masq    1      101        0

TCP  10.25.63.79:2972 wlc persistent 360
 -- 192.168.2.131:1981           Masq    1      10         0
 -- 192.168.2.132:1982           Masq    1      9          0
 -- 192.168.2.133:1983           Masq    1      10         0
 -- 192.168.2.134:1984           Masq    1      0          0

Note that there are only three clustered IIS web servers connecting into the four application server pool, so each web server (and each of its threads) connected to a discrete application server; the web client paradigm typically requires client persistence enabled, so there are no concurrency issues that comes with multi-threaded requests.

Optionally, you will need to setup some iproute2 rules on each of the application services, but only if you want the hosts running those target virtual services to be also reachable by other means, such as ssh’inf directly to the physical hostname.  Here’s how to configure that on a RHEL 5 server.  Modify /etc/iproute2/rt_tables and append:

118    development
128    test
138    production

… and create two new files in /etc/sysconfig/networking-scripts as rule-{adapter} and route-{adapter}, substituting {adapter} with the name of the physical network adapter, i.e., eth0 or br1.  First, the rule file:

from 192.168.2.111 lookup development
from 192.168.2.112 lookup development

from 192.168.2.121 lookup test
from 192.168.2.122 lookup test

from 192.168.2.131 lookup production
from 192.168.2.132 lookup production
from 192.168.2.133 lookup production
from 192.168.2.134 lookup production

Then the route file:

default via 192.168.2.118 table development
default via 192.168.2.128 table test
default via 192.168.2.138 table production