End of day anti-pattern

I made a foolish mistake today. I should have known better... Now it is time to resolve to make new and different mistakes tomorrow, instead of making this same mistake yet again.

Wait For Green

Two of the programs used for our installation only make sense when run as an administrator. The Windows Vista and Windows Server 2008 user account control (UAC) code does not allow administrator privileged users to run programs as an administrator unless they explicitly select "Run as Administrator", or if the program has been marked with a manifest which tells the operating system to always run this program as an administrator.

Those two programs were using the default manifest, and one of the two was adding itself to the "Start Programs" menu. When a Windows Server 2008 user clicked the program with user account control enabled, it crashed painfully and offered to send a report of the problem to Microsoft. We didn't want that.

Visual Studio has an easy way to add manifests to applications, and it seemed simple enough that even a manager could do it...

I made the change, compiled the code, reviewed the code with a real programmer, and let it sit for a few days. This evening, just before leaving for home, I submitted it to the source master and left the building for the swim team dinner.

Don't Do That!

That was a mistake. The compilation failed on the production build machine, even though it had worked flawlessly on my machine. Apparently the production build machine has a different version of one of the compiler components in its path, and that different version did not recognize the options inserted by my version of the Visual Studio IDE. Ugh.

Check the Continuous Integration Server After You Commit!

I've reverted my change, but not after keeping someone else up late into the night because I was not paying attention to the results of my code change.

(There's another thing which is "have fast continuous integration", but we're still working on that...)

"Benefit" that doesn't provide benefit

The U.S. government has allowed "pre-tax" medical reimbursement for several years now. The system allows my employer to take money from my pay before they compute and deduct the personal income tax, social security tax, etc. Those funds are placed in an "account" from which I may request reimbursement for qualifying medical expenses.

The medical reimbursement account has the effect of making some of my health care payments "tax free", stretching my health care funds further by the amount of my tax rate. That is a nice benefit, and I've used it for many years.

As an added benefit, my previous employer and my current employer both offer a Mastercard which can be used to pay eligible medical expenses directly, instead of paying them "out of pocket" and then requesting separate reimbursement.

That Mastercard seems like an ideal solution. It reduces my paperwork and it could reduce my costs by not requiring that I mail evidence for the reimbursement. It could reduce the costs for the benefits provider since they would not have to process the reimbursement evidence either.

But No, There's More (or Less)

Unfortunately, it doesn't work that way. It appears that almost every time I use the "benny card" (Mastercard to pay from the reimbursement account), the provider is required by the government to gather proof of the validity of the expense.

The sequence I wanted was:

1. I pay a medical expense with the Benny card
   
2. The provider pays the expense and deducts the expense from my account
   

The sequence I get is:

1. I pay a medical expense with the Benny card
   
2. The provider pays the expense and deducts the expense from my account
   
3. The provider requests proof of the validity of the expense
   
4. I find the receipt (by this point, several weeks old), copy and mail the copy to the provider
   
5. The provider processes the receipt and decides it is valid (or not)
   
6. If not valid, the provider rejects our claim and request repayment of the money they had paid
   

The actual sequence is worse than using the Benny card, not better!

I'm not clear on the root cause of the problem, but some of the alternatives to this sequence might be:

• Stop requiring validity checks of expenses, accept some fraud as cheaper than the alternative
  
• Declare all expenses from certain providers as "valid" (doctors, pharmacies, etc.)
  
• Stop pre-tax medical reimbursements and either find another way to provide comparable benefit, or admit that the benefit is not valuable enough for the expense it creates
  

My moral: Be careful of unintended consequences. I doubt our elected representatives or the people who planned the pre-tax medical reimbursement system would be pleased that I have decided to never use the Benny card again, because its use is more onerous than the old reimbursement system

Personal Continuous Integration

I've been experimenting for some months with distributed version control system (Mercurial and Git) on various personal projects.

The most interesting of the personal projects was a branch from an active code base that I wanted to extend in a slightly different direction. Because my changes are going in a slightly different direction, they were not likely to be included in the original code base for a very long time. That meant I was going to be "on a branch" from that original code base for a very long time.

Past experiences with CVS and Perforce suggested that being on a branch for a very long time could be a problem. Perforce was not as bad as CVS, but still it was difficult to maintain a separate personal branch and "keep it healthy". Inevitably I would make changes that were harmful to the main code, and not realize I had harmed it because I was not getting all the feedback that is available to developers on the main code.

The main code developers have a continuous integration setup which runs automated tests in many different configurations and summarizes the results to a central location. The results are easy to browse and easy to watch as they evolve.

Since I'm not on the main code, I don't have that nice infrastructure to support my private branch. With distributed version control, I have all the power of a version control system (incremental checkin, revert to previous point in time, branching, merging, etc.). Why can't I have all the power of a continuous integration server for myself as well.

Why Not Use My Own Machine?

I'm sure others have already realized this, but with the ease of installing, configuring, and using the Hudson continuous integration server, I can run my own continuous integration server which compiles and tests code from my personal version control, before it is ever pushed to any other person (or system) in the organization.

Setup Idea

1. Install Sun JDK (needed by Hudson)
   
2. Install Hudson continuous integration server
   
3. Run the Hudson continuous integration server
   
4. Install distributed version control (Mercurial or Git or ...)
   
5. Install Hudson plugin for selected distributed version control
   
6. Clone the development repository from the company central location
   
7. Create a new Hudson job which monitors the local repository, checks out the source from the local repository when something changes, compiles it, tests it, and reports its results
   
8. Start making local changes, checking them in, and enjoying the benefits of continuous integration test runs while developing the code, without the danger of checking into a central repository before the code is "done done"
   

I think those setup steps could be reduced to less than a day, and that day would repay itself within the first 4 weeks of work as developers were more confident in their changes before they shared them with others. I don't yet know if the idea will work for "real" developers, since I'm the manager trying to provide them the tools and environment to be successful, rather than a full time developer. We'll see if the idea actually reduces the feedback loop, and if it actually is viewed positively by the developers. What Will They Value? I think developers will gain

1. Rapid feedback on their changes
   
2. Background work (compiling and testing while they work on other steps)
   
3. Isolation of their changes from others until they are "done done"
   
4. Visibility to the impact of changes from others in their test environment (integrating changes from others onto their branch is a "trigger" event, just as checking in their own changes is a trigger event)
   

CAST 2009 - Early Bird Discount Extended

The 2009 Conference of the Association for Software Testing is being held in Colorado Springs, CO this year. I've registered and am preparing to attend Dr. Cem Kaner's first day tutorial on metrics and how they can serve our stakeholders.

The early bird discount has been extended to May 1. Now is a good time to persuade your manager that a conference with Jonathan Koomey, Jerry Weinberg, James Bach, Cem Kaner, Michael Bolton, Mike Dwyer, and Scott Barber is a great investment for your business.

Five Whys and Four Fingers Pointing Back At Me...

A bug report at work traveled a somewhat strange path as we tried to deduce the root cause of the problem. That strange path reminded me that very frequently when I follow the "Ask Five Whys" heuristic, I discover that there are things which I can change which will improve the situation.

In this particular case, a bug was found, fixed, and then reopened because it was apparently not fixed. There were then several e-mail exchanges between various people as they tried to deduce why the bug was still not fixed. The submitter was confident that the bug was still in the software, so it could not have been fixed. The fixer was confident the change had been submitted, so it must be a problem somewhere else. Others in the conversation wondered if there were additional complications which had not been considered. All of those ideas (and more) could have been correct.

In this specific case, a series of simple gaps were enough to mislead us all.

1. A translation mistake was discovered in late March
   
2. The bug report was assigned to the wrong person, but e-mail exchanges alerted the translation team that the bug existed and needed to be fixed
   
3. In early April the corrected translation was added to the source master
   
4. Just before the corrected translation was added, a new build was generated as part of our once a week schedule of builds
   
5. The submitter tested the fix with the build just prior to the fix
   
6. The e-mail discussion was then started trying to understand why the bug was not fixed
   

When I started asking "Five Whys", I thought it was obvious where the problem originated, and even how to fix it. The bug had been sent to the wrong person, and then when the bug was fixed the bug report was not updated to show which build included the fix.

However, as I stared at the problem further, I realized there was a more significant problem than I had seen initially, and that more significant problem has caused other issues as well.

Why did the fixer need to waste the time guessing which build would include the fix. Couldn't a system tell the submitter when their bug fix was in a build? For example, most bug fixes will reference the bug number in their submit message. Why not pass that information automatically to the submitter, or to the bug report so the fixer does not have to think about the number or name of the next build.

That would have helped, but it appeared that a bigger problem was that the tester did not have easy access to the list of changes which had been made in the build being tested. That list of changes was difficult to find, and difficult to read (I don't find CruiseControl output especially friendly) and probably not known to the submitter at all.

Make Available Information Reachable - Reduce Guessing

When information is not readily available to our very smart people, they will apply skill and judgment and make the best assumptions they can with the information they have. Making that information more readily available will allow them to do their jobs better and reduce wasted work.

The root problem seemed to be someone else's issue, until my thought processes came back to highlight that it was really my problem. I'm the manager, and ultimately it is my fault. Sometimes it becomes obvious more quickly, other times it takes a little more time...

I admit was well that making information easily reachable by those who need it, when they need it, is only part of the answer. That would have helped the tester, but did not help the submitter send the bug to the right person, nor did it help the fixer insert the right data in the bug report. There are so many "why" questions to ask, and so many ways to make small improvements that might help a little.

CAST 2009 - Koomey, Kaner, Weinberg, and Bach

I've done it. I spent the personal funds to attend the 2009 Conference of the Association for Software Testing. Economic times are hard, conferences are difficult to justify to the company, but this conference looks too good to miss.

I've registered for Dr. Cem Kaner's tutorial on metrics and qualitative measurements. Considering the years I've struggled with quantitative measurement in the team, I'd love to hear a different approach.

I've learned plenty from the Association for Software Testing Black Box Software Testing courses ("Foundations" and "Bug Advocacy") and am looking forward to the next course ("Domain Testing"). I'm looking forward to this conference!

(I'm also excited that I'll be able to hang out in Colorado Springs, CO for a few evenings with my oldest daughter. She's an electrical engineer at a Colorado Springs company and has agreed to find a place where I can crash for the night.)

Multi-stake Youth Dance - What Worked, What Didn't

Coleen and I are the co-chairs of the youth activities committee for our stake, a group of 10 LDS congregations in our area. That means we are responsible for a series of events for the youth age 12 to 18.

Multi-Stake Dance

As part of that assignment, we hosted a dance last night for the youth ages 14-18 from Loveland, Longmont, and Greeley. Other youth were welcome to attend (including the 17 who came from Fort Collins, and any who may have come from Laramie or Cheyenne).

The dance was an interesting exercise to plan, host, prepare, present, and repair. In the spirit of "relentless improvement", here are my observations about the dance.

What Worked Great

• The Eiffel tower that Matthew Pond created was amazing. He constructed an 8 foot tall Eiffel tower framework from 2x4 lumber. Dionne Lee covered it with butcher paper and drew squares on the paper to give the feel of the Eiffel tower. In an unexpected twist, many of the youth at the dance signed their names to the tower. Good conversation piece and a good centerpiece.


• The chocolate fountain was a big hit and was surprisingly tidy. Coleen provided platters of rice krispie treats, bananas, apples, and angel food cake squares which the kids could dip into the fountain, then eat as a chocolate coated treat


• Priority Five, an acappella group from Berthoud performed 4 songs, all with the intent that the songs were dance numbers, not performance pieces. The youth didn't seem to dance much, but we encouraged them to dance, and they were thoroughly impressed by a live performance from a first class acappella group.


• A carabiner through an eye bolt is a great way to string lights in the gym, and makes it easy to remove them when the dance is finished

What Worked

• Tourist posters from member photos of the destination (we used "France" as our theme, and used pictures from Brittany West's trip to Paris). Printing the pictures in poster size was easy (upload them to the Sams Photo Club site, order the pictures online, and pick them up), and they looked good


• Use the decorations as prizes. We gave away the posters at the end of the night, some to committee members, some to raffle winners. The youth who received them seemed happy to get them, and we had a few less things to take home after the dance was done.


• Good musical selection and disc jockey work from Christian Dunn. His sound system was excellent, his fees reasonable, and his music selection seemed to fit the youth very well.

What Didn't Work As Well

• Postcards as a mixer activity. We distributed postcards to the youth as they arrived and invited them to have their dance partners sign the postcard. Once the postcard was signed with 7 names, they received a raffle ticket for the end of dance drawing. Some of the youth enjoyed the activity, and the 5 raffle tickets which were chosen all claimed their poster prizes, but it didn't seem to generate much enthusiasm among the larger group of youth.


• Not enough setup and teardown help from the committee, in large measure because the dance was on a Friday night. Their school events and athletic activities compete for their Friday nights even more than they compete for their Saturday nights. I think teardown was also affected by the late hour. The dance finished at 10:00 PM, and we didn't leave the building until almost 11:00 PM.

What Didn't Work At All

• Friday night is the wrong night for an event that requires significant preparation time. Coleen was on the run all day gathering food and supplies for the dance, I was on the run after leaving work early just before 4:00 PM, and still we had youth and adult chaperones arriving at the dance before we were ready.
  
  Saturday is the night for big events, not Friday


• Don't forget the little details (need a checklist of little details to avoid forgetting them in the future)


• Without keys to the church, setup will not happen


• Buy much food early (as much as you can), since the day of the dance is filled with plenty of other things


• Bring tools to the setup in case something needs repair or rework


• Include more people in setup and teardown


• Use an agenda during the chaperone orientation meeting so the meeting runs smoothly