Monday, December 30, 2013

Sequence Based FlexUnit integration testing

Did you know that sequence based testing is built into FlexUnit? I was relieved to find this link in the documentation (http://tutorials.digitalprimates.net/flexunit/Unit-13.html). If you go to that page, please scroll to the bottom section about "Using Sequences".

One of Twin's clients needed a real time update for currency exchange. Basically in our system, instead of a continuous feed, the business requirement was that every week the admin would enter the new exchange rates (there were only 5 that were relevant). Once saved, there was supposed to be a real time push notification to all open clients so that they could update their forecast data. 

Here is what my integration test needed to prove: 
  • Get the current exchange rates from the service
  • Save a new exchange rate
  • Receive the push notification
  • Validate that the new rates has been recorded into model under the correct date.
var rtmpManager:ExchangeRateRTMPManager = new ExchangeRateRTMPManager();
var sequence:SequenceRunner = new SequenceRunner(this);
    sequence.addStep(new SequenceCaller(eventDispatcher, getInitialExchanges, [weeks] ))
    sequence.addStep(new SequenceWaiter(eventDispatcher, INITIAL_RATES_EVENT,TIMEOUT, handleTimeout));
         
    sequence.addStep(new SequenceCaller(eventDispatcher, saveCurrency ,[er]))
    sequence.addStep(new SequenceWaiter(eventDispatcher, SAVE_CURRENCY_EVENT,TIMEOUT, handleTimeout));
  
    sequence.addStep(new SequenceDelay(15*1000)) //Delay 15 s to see if we can get a notification in that time
    sequence.addAssertHandler( validateRTMPManagerReceivedPushNotifcation, null);

    sequence.run();

Tests passed, and it worked beautifully!

Permissions Modelling in Flex

Permissions is something that most of my projects have needed. I've been iterating on a design for many projects, and I've finally come up with something (leveraging some work that my Twin teammates have produced)  that is scalable, testable, and easy for my team to implement. I've written this up in the context of Flex / Actionscript, but there is no reason why it couldn't apply to other languages.

Just to clarify my requirements, here are the user stories that I'm working with. 
  • Should be testable.
    As the developer, I want to be able to declare which permissions are present during testing and development independent of my actual permissions on the system. (In other words - decoupled)  
  • Should be scalable
    As a client, I want to be able to add/remove permissions at any time without major work/rework in the application
  • Easy to understand
    As a new or jr. developer, I want to be able to understand and implement permissions in a best practices way

What I came up with, was a hierarchical string based system. It looks like the following:
public class PermissionTaskConstants
    {
        public static const DEVELOPER:String = "developer"
        public static const ADMIN:String = "admin";
        public static const ADMIN_USER:String = "admin.user";
 public static const ADMIN_CURRENCY:String = "admin.currency"
        public static const LOGIN:String = "login"
        public static const PROJECT:String = "project"
        public static const PROJECT_EDIT:String = "project.edit"
        public static const STAFFING:String = "staffing";
        public static const STAFFING_EDIT:String = "staffing.edit";
        public static const STAFFING_ALLOCATE:String = "staffing.edit.allocate"

    }

In this way, if you were assigned a permission of "staffing.edit.allocate", you would inherit all of the rights of "staffing" and "staffing.edit". This satisfy's the scalable requirement. It is trivial to add or remove permissions, as it would only mean adding / removing strings from the permission path.

These permissions would live in a "permission registry", which would contain an array of strings. This registry could be populated from a service call to a database, or it could be filled at run time. Filling it at run-time would satisfy the "should be testable" requirement. The code is availabe at the bottom, but I've set it up so that it dispatches events when permissions are changed.  A developer can listen for those events and update the UI. Although the typical workflow is that after login, we retrieve the permissions from the database, and the registry is populated before any of the UI is displayed.

IMPORTANT: Decouple the registry from the view!

So this is super important. In order to make this testable and decoupled, never, ever, ever reference the registry in the view. Said differently MXML files should have NO KNOWLEDGE of the PermissionsRegistry.

Here is the recommended workflow. You have a MXML file that represents the view. Just for clarification, this view is in charge of the layout and interactions with the user. There should be no business logic in the view.

Then you have an presentation model (PM) that is in charge of all business logic. This is likely a pure actionscript file. Business logic means conditionals, domain level events, and in this case permissions. The PM receives a copy of the permission registry through dependency injection. The PM then exposes the specific permissions that the view needs through methods.

The view creates its own Bindable public properties that represent the various permission states, and then reads them from the PM. (Please forgive the incorrect capitalization in the MXML objects, the css "brush" for this formatting strips that out)



    <![CDATA[
        import mx.controls.Alert;
        import mx.events.FlexEvent;

        public var pm:PermissionsPM;
        [Bindable] public var canReadProjects:Boolean;
        [Bindable] public var canEditProjects:Boolean;
        [Bindable] public var canReadStaffing:Boolean;
        [Bindable] public var canEditStaffing:Boolean;

        private function creationCompleteHandler(event:FlexEvent):void
        {
            pm = new PermissionsPM();
            canReadProjects = pm.canSeeProjects();
            canEditProjects = pm.canEditProjects();
            canReadStaffing = pm.canSeeStaffing();
            canEditStaffing = pm.canEditStaffing();

        }
        ]]>
    
    
    

package
{
 import com.squaredi.permissions.PermissionTaskConstants;
 import com.squaredi.permissions.PermissionsRegistry;

 import mx.collections.ArrayCollection;

 public class PermissionsPM
 {
  [Inject]  public var permissionsRegistry: PermissionsRegistry;

  public function PermissionsPM()
  {
   permissionsRegistry = new PermissionsRegistry();
   var permissions:Array = [PermissionTaskConstants.PROJECT_EDIT, PermissionTaskConstants.STAFFING]
   permissionsRegistry.permissionsList = new ArrayCollection(permissions);
  }

  public function canSeeProjects():Boolean
  {
   return permissionsRegistry.hasPermission(PermissionTaskConstants.PROJECT);
  }

  public function canEditProjects():Boolean
  {
   return permissionsRegistry.hasPermission(PermissionTaskConstants.PROJECT_EDIT);
  }

  public function canSeeStaffing():Boolean
  {
   return permissionsRegistry.hasPermission(PermissionTaskConstants.STAFFING);
  }

  public function canEditStaffing():Boolean
  {
   return permissionsRegistry.hasPermission(PermissionTaskConstants.STAFFING_EDIT);
  }
 }
}


A couple of other special features about the code. As a developer I can manually add and remove permissions from the list. When I do this, it sets a "developerUpdated" flag, which will ignore setting the whole array. Here is the use case for that. I login to the system but the service call to get permissions is delayed (for some reason). Then I want to verify how the system looks for other users, and add specific permissions, then the service call comes back and could potentially overwrite the values that I've modified. So there is code in place to prevent that.
public function set permissionSource(v:Array):void
        {
          if (developerUpdated) { return} //Don't allow it to be overwritten if the developer has already set individual permissions for testing
          permissionsList.source = v;
          permObj = createPermObj(_permissionsList)
          notifyUpdate();
        }

        public function addPermission(permission:String):void
        {
            var idx:int = permissionsList.getItemIndex(permission);
            if (idx <0)
            {
                permissionsList.addItem(permission);
                permObj = createPermObj(permissionsList)
                developerUpdated = true;
                notifyUpdate();
            }
        }

The other feature that is in the code, is an interface for extracting the data. This could be used in the case of a value object (VO) that returns from the database that has a list of permissions for a given user. This VO is an object, and doesn't have the necessary string representation for permissions. The permission registry accepts an PermissionExtractor helper, that can convert the VO into permissions strings.
package com.squaredi.permissions
{
    public interface IPermissionExtrator
    {
        function getPermissionString(perm:Object):String; // returns a.b.c
    }
}


All of the code and an example can be found at: https://github.com/dshefman/FlexPermissions

Quick and Free, Remote-Team Planning Poker

Recently I was on a hybrid team, made up of representatives from Twin Technologies and 2 other companies. We were all over the country, and we needed a way to play planning poker effectively. Given the other tools that we were using, adding an additional tool, credentials, IT clearance, etc. wasn't feasible.

So we found this great blog post from fourkitchens about using a Google Spreadsheet for planning poker.


To play planning poker, we all call into the phone / voip conference number and open the google doc. After we discuss each story, everyone enters their story point vote, but doesn't press Enter right away. The scrum master can see that everyone has entered a value when their respective box is highlighted grey. Once everyone has made a choice, the scrum master calls “3, 2, 1: Go” and everyone hits Enter. All values appear on everyone’s screen and the planning poker game continues as normal.

We color coded the cells based on conditional formatting (Format -> conditional formatting). Each Fibonacci number got its own color. Invalid numbers remained white. This gave an easy visual representation on the distribution of values.

Now you might have noticed the "Average" line in the photo. We were spending too much time discussing and coming to a unanimous agreement on these estimates.  Based on recent recommendations in the Agile world, everyone doesn't have to agree on a number. The numbers do have to be within a spread of 3 though. For example, 2,3,5 or 3,5,8 are valid spreads. The spread has to be there present though: 2,5,5 or 3,3,8 are not valid spreads and would need to be discussed. (The photo above would represent a invalid spread and we'd have to have more discussion until we could bring that in).

With the adopted averaging the numbers, we would round up to get our estimated point value. Usually what would happen, is that as the team size would unfortunately fluctuate and /or different people could or couldn't attend the planning, we would change what was being averaged. The picture above shows the average only from Rick, Drew, and Larry, and the resulting estimate would be 2 story points (rounding up, or Math.ceil(1.333) :) ) .

Given this system of averaging, it is likely and common to end up with non-Fibonacci totals for our estimates. That is okay. Since the estimates have no time based correlation, they are only used to determine our velocity and therefore our commitment, using Fibonacci only numbers as the inputs and the average as the output works out really well. Fours, sixes, and sevens in out estimate gave us good insight into velocity.

Now just to repeat how agile estimates are supposed to work. The estimate is RELATIVELY sized based on team decided criteria. These UNIT-LESS numbers are used solely to determine velocity. It will likely take 2-3 sprints to even figure out what the velocity is. The estimate HAS NO CORRELATION to time. Once the velocity has been determined, then it is used to make the COMMITMENT. The team says, that we can do (x) story points per sprint, so we will commit and GUARANTEE that (x) will be done by the end of the sprint. That is why averaging works in this case, because the numbers roll into the commitment, and not into any time units.

This system is fast and simple. Adding or removing team members is a no-brainer. Estimating was no longer a chore, nor took up very much time. We could get in and out very quickly and get back to work. I hope that it works as well for you.

Friday, December 6, 2013

Remoting Landmine without a stack trace or debug ability

Summary: 

While using AMF remoting, if you get the following error: 

TypeError: Error #1034: Type Coercion failed: cannot convert Object@1456c7b9 to mx.messaging.messages.IMessage

And you've overridden the  hashcode to your java VO objects, you will end up in an un-debuggable situation. 

Solution:  Remove the hashcode override. 

BackStory:

We are migrating database schemas, so I had set up unit tests to compare the results from one schema to the other. I am using JUnit and assertEquals. 
According to the documentation, I should override the equals() method on each object, as that is what the assertEquals uses for the comparison. This was tedious, but then I discovered that Eclipse had a generate equals() and hashcode() methods. Eclipse comments say to always include hashcode when overriding equals. Ok. I confirmed that on google, all things checked out. 

What happened though, is that serializing from Java to Flex went fine, but on the return trip --- BOOM. 
The error was an UncaughtErrorEvent.UNCAUGHT_ERROR. The request made it out, according to Charles webproxy, but never actually made it to the endpoint on the other side (Based on the debugging breakpoint). Since it never made it to the service, the normal fault responder didn't fire. Fortunately, I'm trapping all uncaught error messages in Flex. There was no stack trace in Flex or in Java. Just this error thrown from somewhere.

The only information I could gather from Charles was that it was a StackOverflowError. 

I only discovered this information by rolling back one commit at a time, then cherry-picking one file at a time, then one method at a time. Painful! 

If you run into this, I hope that I've included enough keywords that you find it quickly and don't waste a whole day like I did. 




Tuesday, August 6, 2013

Software Estimating Tips / Tricks / Traps

This post is broken down into the following areas
  1. The presentation given at
       360Stack on Aug 6, 2013
       Houston TechFest on Sept 28, 2013
       Software Craftmanship North American on Nov 9, 2013
       Agile Leadership Network - Houston on Feb 20, 2014
       PMIHouston - June 8, 2015
  2. The highlights from the presentation, in case you don't want to read the slides
  3. Resources
  4. Questions that were asked during the presentation
  5. Backstory to the presentation
  6. Cross-links
  7. Feedback and comments


Software Estimating: Tips for upping your SWAG

I'd love to hear your feedback

9 Tips for improving your software estimating abilities


  1. Determine if you are supposed to be estimating or hitting a target
  2. Use units that are consistent with the estimates underlying accuracy
  3. Explicitly express the uncertainty in your estimates
  4. Collect and use historical data
  5. Count when possible; compute when you can't count
  6. Do a pre-mortem
  7. Envision someone else (not you) doing the work
  8. Remember the most commonly forgotten project related tasks
  9. NEVER TRY. Do or do not. Manage your commitment and your implied commitment with vigilance. 


Resources:


Questions from presentation

  • Could you talk about larger projects, like bigger than six months. Do these techniques still apply?
     The "Demystifying the Black Art" book is absolutely structured to discuss which techniques work at which stage of the project. It also categorizes by project size, what stage in the project, and type of project (iterative or sequential).
  • How do you build in scope creep into your estimate?
    Put that into your explicit range as a +2 months for 20% new features, or if you are unsure about the client have a +6 month or whatever risk quantification. 
  • Do I charge for estimates?
    I wish clients would pay for it because it is a lot of work, but sadly, no I don't. 
  • Estimating based on historical data... isn't that like your stock prospectus "Past performance doesn't guarantee future performance"?
    Not really, the market is driven by so many factors, but this isn't the case, this is YOUR historical data. Unless your team has had a complete overhaul of people, or you have done a Matrix like download of new skills, your team's historical data is an incredible accurate model to base similar projects against. But beware of the intuitive trap, use concrete numbers (size, number of stories, number of screens, etc) to do the comparison. If you are using intuition or memory for the comparison, you will likely not get a good estimate. 
  • How much effort do I put explaining this [the concepts in this presentation] to the client?
    I make sure it comes out in my language. First I clarify if I'm creating an estimate or hitting a target. If I'm creating an estimate, I make sure that I use estimate and commitment in the same time frame.  Something like the estimate is 5 months, but I can commit to you that I'll deliver something of value, that is less than the whole thing, to you in three months. Give me 2 weeks to get into it, and I can give you a good sense of what that might be. 

    The overall trap, is that you want to avoid committing at the big end of the cone, when it is most uncertain and unclear about what the project is about. 
  • Do you give one estimate to the employees, and an later estimate to the client, so that way the employees finish early?
    Personally, this question made me uncomfortable.  My answer is no, I see no reason to withhold information or give misinformation to my team. As an Agilist, my client is involved all the time. I show them something every week. There is no question to anybody, either on the team or the client, the exact status of the project and if we are on target or not. When my team makes a  commitment, we treat that as such. I can ask my team "I have your word that we will complete all of the things that we say by the time that we said".  If your team is not at a place to give their word, to make that commitment, then it is up to you to determine how you would manage your team and client interactions.
  • Tangent : The research and the book "Drive" by Daniel Pink was mentioned (and I highly recommend it)
  • If estimates are non-negotiable, what happens when you get a new piece of information?
    If you get new information that improves the clarity of the project, and moves you farther into the cone, then by all means, re-estimate. Remember you *should* be re-estimating several times before the end of a project. Estimates are not fixed. 


Backstory on this presentation

I had a great time presenting last year at 360Flex, and I wanted to present again. I knew that since the feel of the conference was changing to be less Flex centric, I wanted to make sure to choose a topic that was generic enough.  At the time, I was working on this crazy estimate with 100+ highly detailed requirements. 

While I was doing this estimate, I thought "I really have no basis for what I'm putting down here.... I'd like to know how to make better estimates" 

Also at the time, I was reading several psychology related books, and the overwhelming consistent information coming from these books, that unless you are an expert at a given task (10K applied hours) that your intuition can easily lead you astray. I was certainly not an expert at estimating and I was completely guessing at what the effort was. 

Therefore, I declared that "Estimating" was going to be my topic, and threw my hat into the ring, so to speak. Immediately I started researching the topic, just in case I got picked. 

The field of estimating is WAAAAAAAY overwhelming. PhD disserations, $100K dollar software packages, complex statistical math, and simple tricks were un-navigably abundant  in my initial research. I decided to narrow my talk to the psychology of estimating; simple things that can to to trick your brain to get you to better estimates. 

I was reading "Thinking Fast and Slow" at the time, and it seemed to fit perfectly with this decision. Half of this mighty book was about professional forecasters and how our environment influences our estimates. It seemed like the perfect match for this presentation.

I started out with an outline based on that book, then I was going to fill it in with details from "Demystifying the Black Art". Once I started reading "Demystifying", that was it... the book was perfectly suited to my presentation. The new outline literally flowed out in an half hour sitting.

It was also fortuitous, because as I was talking to people about the cognitive biases associated with estimating, I got frequent warnings about informing people how poorly their brain works. With these warnings, I was pretty sure that presenting about cognitive biases, it would likely be at best, completely ignored, and at worst, a pretty sizable flame war.

So after several practice presentations to my wife, my mom, the most experienced estimator on the Twin Technologies team, and finally all of  the Twin Technologies team, I have the presentation that is presented to you at 360Stack.




One nice discovery that came about this from this presentation is the "How good of an estimator are you" slide, with the 10 questions. When I was in college, my professor gave us these 10 questions as a quiz. He counted the score as part of the grade, which I wasn't pleased with, as I was average for this test, which was 30%. It was comforting to discover  that 30% is average, even if he didn't mention that fact, and even 20 years later.


Cross links to this blog / presentation

Feedback and comments

  • Fantastic volume of content crammed into a 50 minutes session."
  • Very practical advice on how to estimate projects. One of my favorite sessions."
  • best preso ever!"
  • All around really great! Slides, topic, presentation, speed, q&a, all good
  • The content was well thought out! You did the #1 rule in presenting (tell 'em, tell 'em, tell 'em).
  • Getting access to the slides, opening my eyes to the importance of spending extra time on estimates

Thursday, March 14, 2013

Column Spanning with Flex Spark DataGrid - Part 2

I wrote a prior post ( http://squaredi.blogspot.com/2012/07/column-spanning-with-flex-spark.html ) when I was creating a prototype about column spanning.

I'm going to leave most of the details in that post. I have however put the column spanning into production and discovered some missing logic.

We had a service, that computed user's allocation per week for 2 months that took a long time to run, and we wanted to not lock up the UI while it was running. We wanted a lazy loading solution, that "blocked" out the cells that were not available yet.

Here is what that looked like:

Either scripts and active content are not permitted to run or Adobe Flash Player version 11.1.0 or greater is not installed.

Get Adobe Flash Player

You can hit the button at the top to stop the loading effect.

The latest code can be found at:
https://github.com/dshefman/FlexSparkDataGridSpannableColumns


There were a couple of things that I discovered was missing on ColumnSpanningGridItemRenderer:

  1. Setting the new data to a cell needed to reset the expanded cell
  2. I created a function  called getSpanData() that by default returns data, but could be overwritten if there were nested data structures that was the span data
  3. I added bubbling events, so that the grid could respond to spanning and non-spanning changes as well. 

****Performance Warning*****

There is a performance hit with this spanning solution. You can see it when you scroll. The calculations to determine if spanned and then to reparent and reposition the span takes a toll. Especially if there are alot of spans. For us, that was an acceptable solution, especially since we could rationalize that "it was loading", so of course it is busy doing other things :) 

If you can make it faster, please contribute back on Git. 




Monday, February 11, 2013

Flex GridColumn, complex objects, and default sorting

I recently discovered something about the spark GridColumn. You can pass in a complex type for the dataField.

Say for example that your dataProvider is a collection of Users:

public class User
{
    public var name:String;

    public var id:String;

    public var manager:User;

}


You can specify the gridColumn.datafield as "manager.name"
<s:GridColumn dataField="manager.name" />

This is really cool. Can you can see it in the source code down where the datafield property is set. There is an internal dataFieldPath variable.

Here is the problem that I ran into. I had several columns that had these nested values and I wanted to set up each column to have case insensitive sorting, which is not the default. Using a shared sortFunction was difficult, because I also didn't have direct acces to the dataFieldPath (the nested property), and it would have too much work (in my opinion) to build the object within the sort and then grab the string and then lowercase and compare. It was especially too much work, because it would have negated any label functions that were already in place for these columns.

So here is my solution:


package com.workbook.core.view.datagrid
{
    import mx.formatters.IFormatter;

    import spark.collections.SortField;
    import spark.components.gridClasses.GridColumn;
    import spark.components.gridClasses.GridSortField;

    public class CaseInsensitiveComplexDataPathGridColumn extends GridColumn
    {

        public function CaseInsensitiveComplexDataPathGridColumn()
        {
            super()
        }



        override public function get sortField():SortField
        {
            var sf:SortField = super.sortField;
            const isComplexDataField:Boolean = sf is GridSortField;
            var cF:Function = null;
            if (isComplexDataField && sortCompareFunction == null)
            {
                sf.compareFunction = dataFieldPathSortCompareCaseInsensitive;
            }
            return sf;
        }

        protected function dataFieldPathSortCompareCaseInsensitive(obj1:Object,
                                                                 obj2:Object):int
       {
            if (!obj1 && !obj2)
                return 0;

            if (!obj1)
                return 1;

            if (!obj2)
                return -1;

            const obj1String:String = itemToLabel(obj1).toLowerCase();
            const obj2String:String = itemToLabel(obj2).toLowerCase();

            if ( obj1String < obj2String )
                return -1;

            if ( obj1String > obj2String )
                return 1;

            return 0;
        }


    }

}

What this does, is that when you click on the column header, it grabs the sortfield from the gridColumn (that was the easiest public property to get at). Fortunately if the dataField is complex, it builds a GridSortField object instead of a SortField.

I check for the GridSortField and if there is not already a sortCompareFunction, I assign it to my case insensitive sort function.

Then within the sort function, I call itemToLabel, which fortunately does all of the heavy lifting to determine the nested object's label, calling any labelFunctions as needed.

Then I force to lowercase, and done.

To use, I replace my GridColumn in my dataGrid definition, with CaseInsensitiveComplexDataPathGridColumn.

Wednesday, January 2, 2013

Drive - Book Quotes

I'm starting a new trend, that as I read books, I'm going to list out significant or relevant quotes that I'd like to remember.

Drive

by Daniel Pink:

Something cool about this book is that at the end, there is several levels of summary, from a 140 character twitter summary, an elevator and cocktail party summary, as well as a couple of paragraphs on each chapter.

External motivation ( rewards and punishments) == bad

  1. Extinguish intrinsic motivation
  2. Diminish Performance
  3. Crush Creativity
  4. Crowd out good behavior
  5. Encourages cheating, shortcuts, and unethical behavior
  6. Addictive
  7. Fosters short - term thinking

The Three Elements of intrinsic motivations

  • Autonomy

    • Autonomy over the task
    • Autonomy over their time
    • Autonomy over their technique
    • Autonomy over their team
"The most autonomy crushing mechanism imaginable: the billable hour. Most lawyers must track time, often in six minute increments. If they fail to bill enough hours, their jobs are in jeopardy. As a result their focus inevitably veers from output of their work (solving the clients problem) to its input (piling up as many hours as possible). "
The billable hour makes sense for routine tasks, but for non-routine tasks, the link between how much time somebody spends and what that somebody produces is irregular and unpredictable.
The antithesis of the billable hour is the ROWE (Results-only work environment) 

  • Mastery

    • Mastery is a Mindset
      • It is more like strength then height. You can work to improve it. It is in your control
    • Mastery is a Pain
      • "The mundanity of excellence".. spending the most time and effort on the little details that show the tiniest of improvements
    • Mastery is an Asymptote
      • Mastery is impossible to realize fully, because you can always get better. The joy is in the pursuit more than the realization. In the end, mastery attracts precisely because master eludes. 

  • Purpose

    • The yearning to do what we do in the service of something larger than ourselves

Toolkits

Intrinsic Motivations for Parents:


  • Offer praise... the right way
    • Praise effort and strategy, not intelligence
    • Make praise specific
    • Praise in private: Praise is feedback, not an award ceremony.
    • Offer praise only when there is good reason
  • Help kids see the big picture