Refactoring, the clients perspective

I'm in the middle of a refactoring project, but it doesn't involve any code, and I happen to be the client.

In the process of re-insulating the attic, we discovered that the wiring and electricity in our house is FUBAR. So our electrician is redoing all of the wires correctly. This is a refactoring effort, because after tons of effort, hopefully there will be no difference in functionality. The only difference should be less buggy, more maintainable, more consistent electricity, and the ability to renovate.

Let me explain. We had a hunch that there was something wrong with the electricity, has we would often get a short in the master bath light. The bathroom vent fan wasn't working (a known condition when we bought the house), and wanted it fixed before we covered the attic with insulation. Upon inspection, the wiring for the vent was incredibly non-standard and very complex and took about 3 days just to figure out how it was working (sounds familiar so some of the  classes that I've run across). Further inspection showed that most outlets weren't grounded, and most of the junction boxes where overloaded (you can think of this as violations of the single responsibility principle). There was bare wire and extension cords running throughout the attic.

I'm not an electrician, but even I can figure out that this is bad.

So after nearly 6 weeks of banging around the attic, the project is nearly done. And there is barely anything to show for it. Granted, we did add some feature requests (who doesn't?)....
Things like, we could really use an outlet here, or these outlets don't have a ground (known), can we change that. etc. But for 6 weeks of work, we essentially are getting a third hole in an outlet receptacle, and a working vent fan in the bathroom.

So having previously successfully sold a refactoring project, previously failed at selling a refactoring project where it was desperately needed, and now as the recipient of a refactoring project, I feel like I have some interesting insight to share.

Some important background information:

• I trust the electrician. This is important. I believe him to the be expert and have my best interest first. 
• He is exceptionally meticulous and honest. 

So here are some of my frustrations about the process.

• It took about 3x longer then expected
• Even though I knew work was happening, I had no sense of progress
• When I did get a progress report, I had no idea what the electrician was talking about 

Here are the things that I found most helpful.

• There was an "all known" point. This was the point where he had figured out everything: what each wire, junction, and switch did. There were no more surprises after this point. It was at this point that we could start committing to deadlines. (Just as a note, we didn't get to this point until after the original estimate had elapsed). We weren't done, not even close, but we knew everything there was to know and could easily layout the road map for the work to get done.
• We started a daily list of what he had done that day, what he was planning on getting done the next day, and overall tasks that he knew where still outstanding. For the most part, I didn't understand what most of it meant, but I was able to at least follow along with what he was working on and what he still had left to do. (When did I suddenly become the annoying project manager wanting to check off rows of tasks in Excel? Actually I wasn't applying much pressure, I just wanted to see when I could schedule the insulators, because I was hoping to do it before the end of the year)
  
  This list became our "Stand up" - what happened yesterday, what is you plan for today. As well as a work in progress (WIP) account. Because the detailed tasks were linked to overall larger tasks and goals.
   
• There were several mini visual milestones... oh this light is working again, or this outlet is now grounded. It wasn't much, but at least there was something to see / touch.
• There was something satisfying about seeing large bags of trash being removed from the attic.

  Here are some outcomes to think about:

• Estimate only up to the "all known" point.  So only plan to work up through the learning part of the code. In terms of refactoring, I think you could rearrange functions within a class, rename private methods and variables, and essentially tag / comment sections and files as to what they do. You could also establish characterizations tests for the pieces that are testable.
  
  Do not estimate the entire scope of the project until you get to the "all known" point.
• Figure out beneficial ways to show progress. Some of the metrics that I've used in the past is
• # of methods under test
• # public methods and variables (shouldn't change much for a refactoring) 
• # private methods and variables (maybe should decrease to improve testing and extending)
• # protected methods and variables (should increase, more methods probably mean smaller, easier to read methods)
• # of import statements (decreasing shows less dependancy)
• PMD Errors 
• PMD Warnings
• # average method length
  
  (Note, I'm still looking for some static analysis tool that gets a metric for code readability. Anybody know of any? )
• Clearly and in detail list all of the work that you've done. Your VCS comments and diff log can help you with this.
• Engage the customer in a non-time consuming way about what your are your plans large and small (next hour, today, this week). One of the frustrating things about our processes, was the electrician tried to tell me all of the stuff he was doing? This felt like a waste of time, as I kept hearing the same things over and over, I was much happier when I could see it on a sheet of paper, and we saved the discussion for critical decisions (I have to make a change to this fixture or outlet. I'm not able to keep it the same, what would you like me to do). Otherwise, I'd smile and nod, but not really understand  if what he was saying was important to me ( Is there something I need to know) or not.
• Showing progress on a refactoring effort is difficult. I would say # of classes left to refactor is a mini-visual milestone. I'd gladly take other comments on ways of showing progress for a refactoring effort.
• I think credibility is important too. Our electrician kept saying how much better it was and that it was good that we were doing this. I found this frustrating. It wasn't until we went up into the attic midway and saw the difference did we recognize the value. But he had to get it to a point where we could see it, we weren't going to crawl back into the depths of the attic (that is why we hired him). So if your refactored code is much easier to understand, even for a non-technical person, there might be value in showing it (if that person even values looking at code. Alot don't - their eyes glaze over if it comes near). If there is another developer nearby (not even from the same language, team or area... a backend dev, or dba, or someone else technical) that can look at the before and after and at some validation. "That looks much better..... I don't even know what I'm looking at, but the newer stuff is much cleaner and more understandable."
• When we started going over budget, the electrician gave variable rates. There was one rate for the difficult in the attic work (it is uncomfortable and cramped) and a lower rate for the easy on the ground work. I'm not sure if this can apply to coding, but it was a nice gesture on his part. 

So there you have it. I've tried to share as much as I could about this experience as it relates to refactoring. Even when knowing the process, it is hard not to be frustrated. It is hard to not think to yourself, "He is *just* adding a grounded wire to an outlet that is already there. How hard can that be? Why is this taking so long".  I have no idea about how to do what he is doing, but it seems like it would be easy. Obviously it is not.

I'm sure many people say the same thing about coding. "Certainly adding a button in the middle of the datagrid can't be *that* hard. How long does it take to add another page to the site, the menu is already there, just make some more room at the end."

I'm just glad that I trusted our electrician. This might have been a different story had I not. 

Recycled Keyboard Ornament for Geeks / Programmers

So, as you can tell by my posts, a large part of my year has been working with the Flex DataGrids in their various flavors.

Well, my wife and I have a tradition of choosing an ornament for each of us that represent our year. My wife did a fabulous job creating this christmas ornament for me out of a broken keyboard.

Made from keys, the conductive plastic layer that the keys connect to, hot glue, and play-doh :) (Its great to have to have kids around so all that these supplies are readily on hand :) )

Finding a mentor - the wish list

So I'm on a mission to find a mentor or mentors. I guess in order to find something, I would need to figure out what it is I'm looking for (which reminds me of the Nanny wish list from Mary Poppins in some ways)

So ideally, a mentor would be someone who:

• Similarly, but hopefully more, experienced then I am in software development
• Has a modicum of time to dedicate to me. I don't want it to take up much time, but I also recognize that it takes some
• Is willing to share their experience, make suggestions, and has an altruistic interest in helping others on their journey
• Checks in, unprompted, every once in a while, just to maintain connection

Concrete potential ideal examples of interactions from mentee to mentor

• I have a challenging/critical software / architecture problem and a potential solution or solutions. I could ask the mentor which approach might be best. They could say "Have you thought about this", or "what about this scenario", "Does this take 'x' into account", "Have you checked out 'x', it solves a similar problem", etc.
• Being able to show someone, a well designed section of code, and get some constructive criticism (if deserved), or a nice job (if deserved).
• Being able to recommend resources, like conferences, blogs, books, people, etc that are interesting and relevant to  me and my journey.
• Can connect me to potential people who need to be mentored but are not as far along as I am, so I can be a mentor as well. 

Concrete potential ideal examples of interactions from mentor to mentee

• Share pieces of their work, that they are proud of, and explain their thought process, challenges, etc, to someone who might understand
• Ability to talk at a detailed / advanced level to a peer (It is always fun talking about something that you are passionate about to other like minded people
• Material for blogs, presentations, publications that could be helpful for the mentor's journey.
• Hopefully a handful of ah-ha moments :)

Please comment about what you would want from a mentor or mentee. 

Upcoming Presentation @ 360Flex 2012

Title: Decoupling, Refactoring, Testing and Other Improvements to your Craft Topic:

"I'm scared of that file, it does EVERYTHING!"

"That file has changed 77 times in the last month, I have no idea about what it is doing"

"This file touches everything, I'm not changing it" 

Do you ever feel trepidation about modifying a file? Do you fear that making a change might break something and you'd have no idea? This session is about from-the-trenches-experience refactoring, breaking dependencies, and getting tests in place. This is NOT the idealistic Test-Driven Development (TDD) that we all want to do, but how to add characterization tests after the fact - after the code has decayed, and is reeking of code smells. Learn about the essential resources, plug-ins, frameworks, and methodologies that you need to have to regain control of your code and your sanity. 

http://www.360flex.com/   April 15th-18, 2012

Journey of a software craftsman - the backstory

Just as a disclaimer, I want to state my definition of a software craftsman, as someone who recognizes that there is an art, science, and craft to programming and I'm striving to constantly improve towards mastery, knowing that I'm only going to asymptotically approach it. 

So with that out of the way, I'm going to start a small series related to my journey as a software craftsman, finding a mentor, and improving my craft. Before I get to the present, I feel it necessary to record many of the milestones and moments that have brought be up to this point. 

This post is not technical nor specifically related to Flash or Flex, but it does (hopefully) describe the initial part of my journey (before I even recognized I was on one)

• Mid 1990's
• Context:
  I was fresh out of school working at a multimedia company. One of our clients used Macromedia Director to deliver their quarterly+ shareholders meetings. This client had a franchise model, and as part of their presentation wanted photos of each of the franchise buildings. Specifically in their "slides" there were 4 photos spots available, each of a different franchise, and as you advanced forward, the photos shifted by one, so that it looked like a single new photo got added to each slide. This is all well and good, but the number and order of the slides changed constantly (even up to minutes before they gave the presentation.) (As I side note, they often took us along to the meeting so that we could update the stock price to literally be the current price when they presented it). Sometimes there would be 100 slides, sometimes only 15. Oh and also their was a print option, which included the scrolling photos as well.
  
  So basically there was about 150 potential slides, any number of which could be in a given presentation in any given order. Some of the slides had dynamic data, like the stock price, which was updated (manually) all the time.
• Result
  While it didn't have a name to me at the time, through this project I had already separated the functionality (ordering the slides and photos) from the UI (the next frame and printing) and the business rules (externalize everything to text files (no feasible DB or internet available at the time)). It was my discovery of MVC, nearly a decade before I discovered it had a name
•  Late 1990's
• Context
  Many of our projects at the time were training animations with mostly linear controls. I quickly realized that testing the latter parts of the projects was very tedious as you'd have to wait for all of the previous parts to finish.
• Result
  Modular design. I often rigged in temporary navigation panels, debugging tools, and other test harnesses, to allow me to test faster and more accurately. I was able to pull out the pieces from the rest of the application and exercise them. Nothing automated yet though, not quite. About this time I wrote a utility that would crawl the application display list,  find everything with an interaction on it, and force that interaction to fire randomly. This is the equivalent to clicking everything on the screen as fast as you possible can.
  
  This is close to automated testing, but it wasn't repeatable, as it randomly choose what to interact with. But while watching it, and the logs, I discovered many many bugs before it went to production.
• Early/Mid 2000's
• Context
  I had this intense project where the goal was to fully (nearly) simulate a about to be introduced cell phone (with a camera :) ). The project had a one month time frame, and I had absolutely no idea how to do it. Another motivating factor, was that if we could do it, there would potentially be dozens of more phones to simulate. Since there was only a month to deliver the whole thing, there was a demo every Friday. Without having heard of Agile, I was now in my own self defined agile methodology.  At each demo, we did some planning about what would be delivered the following week. It was at this moment, that I realized that the structure that I had put in place didn't scale, and I wouldn't be able to accomplish next week's goal. I would spend the weekend rewriting everything that I had to support the new objectives. This happened EVERY week. Of course each rewrite / refactoring got faster and better
• Result
  I discovered Agile. Now it wasn't formalized, as I wouldn't put a name to it until many years later. I discovered the power of short iterations, not designing upfront, the value of starting over with leveraged experience, and refactoring. This was probably one of my most powerful experiences.
  
  After successfully delivering this project, and the next 2 dozens phones*, I researched how might I do simulations better next time. I came across this book called Flash MX for Interactive Simulation, which talked about State Machines and the State Design Pattern.
  
  This book opened up the world of Design Patterns and UML to me. The coolest thing, was that my solution for the phone WAS INDEED the State Design Pattern. I was ecstatic and amazed that I had "invented" the exact thing that others were practicing.
  
  This was incredible fuel to me. I read everything I could about Design Patterns and UML
• *
• Phone #1 = 1 month, 
• Phone #2  =  2 weeks, 
• Phone #3 = 1 week
• Phone #4+ = 2 days
• Mid/Late 2000's
• Context
  I had the opportunity to be on some teams where there was more then me as the one developer on the team. Some were agile, most were not. I learned the value of source control, integration headaches, testing, and working with other peoples' code. I stated recognizing that code readability was critical, as well as the entire team dynamic, philosophies and varying still levels. (What do you mean you are going to reach up from the child, to the parent, to the grandparent, to another descent and change a property!! #@%! Haven't you ever heard of events? !)
• Result
  Started on the Pragmatic Programmer and Agile philosophies
• Late 2000 - Recently
• Context
  I was on a massive project. Two years, 24 developers. It was huge. I was the Flex technical lead and architect. I had laid everything out (architecturally), both code wise and processs wise. We had a 13point definition of done. But when all said and done the team only accomplished 2 of the 13. We were functionally complete, but we weren't done. Because of several factors, team & management mentality, on-boarding and delegation process... we ended up with what can most easily be described as Legacy Code.
  
  I tried often to make course corrections to the code base and team, but it didn't happen to my satisfaction. This sent me on a massive effort to try to figure out why I couldn't communicate all of my experience to the rest of my team and what I could do to fix it.
  
  I devoured references like Working Effectively With Legacy Code, Clean Code, Clean Coder, Pragmatic Thinking And Learning, and dozens of other books. I was starving for any information about how to communicate better, clean up the code that was there, help my team improve, and figure out what I did wrong, or what I could do better, either immediately or in the future.
• Result
  Recognized that I am a software craftsman, but I still have a lot to learn to get to where I want to be. I am actively working towards becoming a craftsman, becoming a mentor, finding a mentor, improving my craft and trying to advocate that quality is important for my industry. 

If you made it this far, thanks for reading my story and sticking around :) I plan on actively contributing to this topic as I continue on my journey. 

Flash is NOT Dead - communicating to lay people

So, there is lots of chatter around Adobe's announcement today, that I want to add my 2 cents on. I happen to be teaching a class on Flash tonight, so I thought about how to explain this announcement to my class.

While I agree with Adobe's decision (somewhat), I think they royally screwed up their communication of their message.

They've spent years training people to recognize the Flash brand, and failed to realize that most people can't distinguish all of the pieces that make up that brand. Nobody outside of the community knows that there is even a Flash Pro, Flash Builder, Flash Player, AIR, Flex... it is all Flash to them. Adobe has done a good job in that regard. So today's message that says Flash {insert ANY other words here} is not going to be supported, gets interpreted as the entire brand is not going to be supported.

Here is a snippet of their message:

We will no longer continue to develop Flash Player in the browser to work with new mobile device configurations (chipset, browser, OS version, etc.) following the upcoming release of Flash Player 11.1 for Android and BlackBerry PlayBook.

And here is what the world read:

 We will no longer continue to develop Flash Player


That is akin to reading a hypothetical message that says:
"Continental Airlines will close its Houston hub" as
"Continental will stop flying"

This is only a small piece of the whole equation. But this is what the lay person hears... and hence all of the panic today.

I came up with the non-technology analogy to explain it to my class.
(Bare with me, taking the analogy away from technology requires some leeway) :

Imagine that you are a company, called StoreYourStuffCompany, that produces magic spells (Harry Potter references to follow). Your main product is an enchantment, StuffYourStuff, that makes all of your stuff, like books, clothes, furniture, etc, fit into a given space.

Now, the enchantment has been around for a while, and actually StoreYourStuffCompany has partners including home builders, so that your home comes designed with this StuffYourStuff enchantment. Hence all of your stuff fits into your house or apartment.

Recently, people want to carry more stuff with them, and so StoreYourStuffCompany has partnered with backpack and purse manufacturers, like the following videos.


Harry Potter's Undetectable Extension Charm

and Mary Poppins Carpet Bag:

Now the problem, when you have all of your stuff in one of these bags, is that it is poor experience as Mary Poppins shows looking for her tape measure. It is difficult to get to the thing that you are looking for, and you often have to touch everything to find the thing you want.

Yet, in your house finding your stuff is a relatively good experience (assuming a base level of home organization).

So StuffYourStuff comes up with an additional enchantment, GetYourStuff, which in Harry Potter language is the summoning charm, Accio. This charm allows you to simply say what you want and it snaps to your hand if it is nearby.

The GetYourStuff is hugely successful, so much so that people don't carry the backpacks and purses, instead use StuffYourStuff to keep all of their stuff in their pockets, and GetYourStuff brings them the exact items that they need when they need it.

StoreYourStuffCompany decides to discontinue partnering with the backpack and purse makers to heavily focus on improving the GetYourStuff charm, as well as the StuffYourStuff enchantment.

Rephrased for techology this story reads like this:

StoreYourStuffCompany = Adobe
StuffYourStuff = Flash
Home / Apartment = DESKTOP browsers
Backpacks and purses  = MOBILE browsers
GetYourStuff = AIR for desktop, mobile, other

When you are on your phone, do you go to facebook.com or the facebook app? twitter.com or TweetDeck? gmail.com or the email app?

I'm guessing you said the app. That is because the experience is better. You get the information that you need/want faster without touching everything. You go to the web to look up information that you can't get from an app. You are there to accomplish a specific task, when you are browsing on your phone, and HTML5 might be the technology that will allow you to succeed faster?  

Here are some other good insights into the matter:
http://mattgemmell.com/2011/11/09/adobe-communication/
http://forta.com/blog/index.cfm/2011/11/9/Some-Thoughts-On-Flash-And-Devices

My biggest fear is that for the lay people and business people who only hear that "Flash is not to be supported" / "Flash is dead", will base their financial decisions on that little and incorrect sound bite.

Adobe will need to do some serious marketing to get the right message across.

Working effectively with novices

So I have two books that I highly recommend.

First let me explain where I'm coming from: First I'm a professor, mentor, and trainer, and I always have to try to express my knowledge about Flash and Flex to people who are brand new.

Second, as a consultant, architect, and developer, I frequently need to express the value of TDD, Agile, CI, and lots of other really good things, that are not quite development.

These books highlight the differences between a novice's brain and an expert's, and how to start bridging that gap.

Pragmatic Thinking & Learning by Andy Hunt, talks about the novice versus expert brain. How the brain interprets and responds to signals. This is written by programmers, and is incredibly useful.

For me, this book answered the question of why did my project managers always seem to make decisions that I didn't agree with and I urged against.






Made To Stick by Chip & Dan Heath, is a fantastic book about what makes an idea or concept "sticky". Why do we remember certain stories, advertisments, urban legands after hearing them once, yet things critical to our job we forget the moment we leave the meeting.

I feel like this book would have been helpful for me to persuade the aforementioned managers of why some of the things I was staying was important.




For me, these two books are game changers. I re-written my entire University of Houston, "Intro to Multimedia" course to target the novice brain better, and I'm working on incorporating more sticky ideas too.

Because I'm so excited by these books, I've had to share.