Refactoring, the clients perspective

I'm in the middle of a refactoring project, but it doesn't involve any code, and I happen to be the client.

In the process of re-insulating the attic, we discovered that the wiring and electricity in our house is FUBAR. So our electrician is redoing all of the wires correctly. This is a refactoring effort, because after tons of effort, hopefully there will be no difference in functionality. The only difference should be less buggy, more maintainable, more consistent electricity, and the ability to renovate.

Let me explain. We had a hunch that there was something wrong with the electricity, has we would often get a short in the master bath light. The bathroom vent fan wasn't working (a known condition when we bought the house), and wanted it fixed before we covered the attic with insulation. Upon inspection, the wiring for the vent was incredibly non-standard and very complex and took about 3 days just to figure out how it was working (sounds familiar so some of the  classes that I've run across). Further inspection showed that most outlets weren't grounded, and most of the junction boxes where overloaded (you can think of this as violations of the single responsibility principle). There was bare wire and extension cords running throughout the attic.

I'm not an electrician, but even I can figure out that this is bad.

So after nearly 6 weeks of banging around the attic, the project is nearly done. And there is barely anything to show for it. Granted, we did add some feature requests (who doesn't?)....
Things like, we could really use an outlet here, or these outlets don't have a ground (known), can we change that. etc. But for 6 weeks of work, we essentially are getting a third hole in an outlet receptacle, and a working vent fan in the bathroom.

So having previously successfully sold a refactoring project, previously failed at selling a refactoring project where it was desperately needed, and now as the recipient of a refactoring project, I feel like I have some interesting insight to share.

Some important background information:

• I trust the electrician. This is important. I believe him to the be expert and have my best interest first. 
• He is exceptionally meticulous and honest. 

So here are some of my frustrations about the process.

• It took about 3x longer then expected
• Even though I knew work was happening, I had no sense of progress
• When I did get a progress report, I had no idea what the electrician was talking about 

Here are the things that I found most helpful.

• There was an "all known" point. This was the point where he had figured out everything: what each wire, junction, and switch did. There were no more surprises after this point. It was at this point that we could start committing to deadlines. (Just as a note, we didn't get to this point until after the original estimate had elapsed). We weren't done, not even close, but we knew everything there was to know and could easily layout the road map for the work to get done.
• We started a daily list of what he had done that day, what he was planning on getting done the next day, and overall tasks that he knew where still outstanding. For the most part, I didn't understand what most of it meant, but I was able to at least follow along with what he was working on and what he still had left to do. (When did I suddenly become the annoying project manager wanting to check off rows of tasks in Excel? Actually I wasn't applying much pressure, I just wanted to see when I could schedule the insulators, because I was hoping to do it before the end of the year)
  
  This list became our "Stand up" - what happened yesterday, what is you plan for today. As well as a work in progress (WIP) account. Because the detailed tasks were linked to overall larger tasks and goals.
   
• There were several mini visual milestones... oh this light is working again, or this outlet is now grounded. It wasn't much, but at least there was something to see / touch.
• There was something satisfying about seeing large bags of trash being removed from the attic.

  Here are some outcomes to think about:

• Estimate only up to the "all known" point.  So only plan to work up through the learning part of the code. In terms of refactoring, I think you could rearrange functions within a class, rename private methods and variables, and essentially tag / comment sections and files as to what they do. You could also establish characterizations tests for the pieces that are testable.
  
  Do not estimate the entire scope of the project until you get to the "all known" point.
• Figure out beneficial ways to show progress. Some of the metrics that I've used in the past is
• # of methods under test
• # public methods and variables (shouldn't change much for a refactoring) 
• # private methods and variables (maybe should decrease to improve testing and extending)
• # protected methods and variables (should increase, more methods probably mean smaller, easier to read methods)
• # of import statements (decreasing shows less dependancy)
• PMD Errors 
• PMD Warnings
• # average method length
  
  (Note, I'm still looking for some static analysis tool that gets a metric for code readability. Anybody know of any? )
• Clearly and in detail list all of the work that you've done. Your VCS comments and diff log can help you with this.
• Engage the customer in a non-time consuming way about what your are your plans large and small (next hour, today, this week). One of the frustrating things about our processes, was the electrician tried to tell me all of the stuff he was doing? This felt like a waste of time, as I kept hearing the same things over and over, I was much happier when I could see it on a sheet of paper, and we saved the discussion for critical decisions (I have to make a change to this fixture or outlet. I'm not able to keep it the same, what would you like me to do). Otherwise, I'd smile and nod, but not really understand  if what he was saying was important to me ( Is there something I need to know) or not.
• Showing progress on a refactoring effort is difficult. I would say # of classes left to refactor is a mini-visual milestone. I'd gladly take other comments on ways of showing progress for a refactoring effort.
• I think credibility is important too. Our electrician kept saying how much better it was and that it was good that we were doing this. I found this frustrating. It wasn't until we went up into the attic midway and saw the difference did we recognize the value. But he had to get it to a point where we could see it, we weren't going to crawl back into the depths of the attic (that is why we hired him). So if your refactored code is much easier to understand, even for a non-technical person, there might be value in showing it (if that person even values looking at code. Alot don't - their eyes glaze over if it comes near). If there is another developer nearby (not even from the same language, team or area... a backend dev, or dba, or someone else technical) that can look at the before and after and at some validation. "That looks much better..... I don't even know what I'm looking at, but the newer stuff is much cleaner and more understandable."
• When we started going over budget, the electrician gave variable rates. There was one rate for the difficult in the attic work (it is uncomfortable and cramped) and a lower rate for the easy on the ground work. I'm not sure if this can apply to coding, but it was a nice gesture on his part. 

So there you have it. I've tried to share as much as I could about this experience as it relates to refactoring. Even when knowing the process, it is hard not to be frustrated. It is hard to not think to yourself, "He is *just* adding a grounded wire to an outlet that is already there. How hard can that be? Why is this taking so long".  I have no idea about how to do what he is doing, but it seems like it would be easy. Obviously it is not.

I'm sure many people say the same thing about coding. "Certainly adding a button in the middle of the datagrid can't be *that* hard. How long does it take to add another page to the site, the menu is already there, just make some more room at the end."

I'm just glad that I trusted our electrician. This might have been a different story had I not.