Post-Project Analysis: A Strategy for Performance Improvement

Your department has just finished a big project.  Your team members did well, but did not perform perfectly.  You are tempted to briefly talk to the members who need a little feedback, pull the plug, and call it a day.  It's Miller time, right?  Not if improvement is your goal and you want your team to do even better on the next project.  For that to happen, it's time to gather the troops and analyze what happened.

Going over the results of the project when it is complete is not new.  Businesses go over project or sales campaign results with the sales force.  In education, superintendents and principals review and share achievement data with teachers.  Air Force pilots, however, due to the fact that their lives depend on their performance as well as the performance of their teammates, take after-project analyses to new levels. Its a model all of us should follow.  They call them debriefings.

Air Force debriefings begin with certain ground rules and have a specific structure.

The Ground Rules

1. Rankless/nameless - All participants in a debriefing have no rank.  The only person who has any authority is the flight lead (chair or project manager).  Some Air Force squadrons have team members remove their Velcro-attached name tags before entering the debriefing room to reinforce the concept that they don't use names or ranks.
2. No fear of reprimand - There are no pay-backs for what is said in the debriefing.
3. Mission participants only - If someone didn't participate in the project, they are excluded from the debriefing.  This is a team thing.

The ground rules ensure that everyone can speak up and say what needs to be said.  They encourage dialog and put everyone on an even playing field.  It is important to remember that a team is only as strong as its weakest member.  If the weakest member outranks everyone else, these rules ensure that the dialog in the room is authentic.  Although this is not a ground rule, I'll add one more -- it must be done as soon as possible while everyone's memories are fresh and everyone is present; waiting for the next day is a waste of time.  In the Air Force, the debriefing happens about 10 minutes after the mission.

Debriefing Structure (Formatted for us civilians)

1. Review the objective - All discussion should revolve around the objective, as the objective should be the measuring stick by which the project success gauged.  Sometimes, the lack of success is caused by decisions, a lack of goal clarity, or poor communication at the start by the project manager.
2. Start with self-analysis - With the project manager beginning by critiquing his/her own performance, the message is sent that everyone is held accountable, not just the troops in the trenches.  Then the project manager goes around the table soliciting feedback from each member on his/her (the project manager's) performance.  Not just generalizations, but specific things, good and bad, that contributed to the project's success or failure.  Usually general comments will be made at first, but the manager should continue questioning each person until the fine, negatives begin to appear.  Although it's nice to hear subordinates say nice things about one's performance, improvement comes by dealing with the negatives, no matter how much it stings.
3. Revisit the original project plan - The original plan, if it was good, outlined the outcomes that were to be achieved by the project and specified the measurable criteria by which success would be measured.  Making sure that everyone remembered the outcomes and success criteria is the first step.  If it is determined that there were unasked questions at the start of the project that caused a lack of clarity among the team members, one major point of improvement has been found.  Then it must be determined where the miscommunication originated and why clarifying questions were not asked.  If everyone was clear on the specifics and expectations, then the project manager must look at whether the project was too complex for this team's skill sets.  This may cause the project manager to review how he/she selects team members for projects.  It may also reveal a need for staff development.
4. Analyze what happened - An Air Force squadron debriefing would then continue with video tapes and replays of radar data being reviewed.  If four jets were involved, all four tapes would be time-synched and played at the same time.  Mistakes would be highlighted and discussed.  Although nameless, everyone must still account for mistakes.  Usually, everyone makes mistakes and no one comes out of a debriefing without some scrapes and bruises on their ego.  In the business world this would mean reviewing the data, which may include sales figures, profit/loss statements, etc.  In education, this would include achievement data disaggregated by race, grade level, gender, and poverty-level.  Everyone would account for mistakes and deficiencies as well as successes.
5. Review the project outcomes - It's time to come right out and say which success criteria we met and which we didn't. 
6. Generate a "Lessons Learned" sheet - List what was gained from the debriefing -- the lessons that the team learned.  What would the team do differently next time?  What would they do the same?  No names are attached to the sheet as it is then posted for other teams to learn from so they don't have to reinvent the wheel.  We are all more successful when we contribute to the learning and success of others.
7. End on the positives - No matter how dismal the results of a project are, there will be bright spots.  If anything, one can always say that, with what was learned in the debriefing, the team will grow and improve.

Okay, now it's Miller time.

Want More Information?
Source:

Comments

[Art Gelwicks]

The idea of nameless/rankless is an excellent one, although often difficult to achieve if project sponsors who were involved in the project are included in the debriefing. It's possible for a project team to compile a list of recommendations and learnings from the project and then not have them adopted or implemented by "the powers that be". I feel that these kinds of learnings are most effective when implemented at a "low altitude" within the next project and make them an inherent part of the process rather than a banner waving, "Hey! Here's what we learned!"

[Bert]

Yes, I agree that the lessons learned be shared only with other teams *at the same level* as the debriefed team.

James Murphy, author of the book, deals with the rankless issue in detail as he shares that teams in the Air Force can include Generals, who outrank all other team members but who may be the poorest performing member of the team on that particular mission. All members of the flight are designated by numbers (Number One, Two, etc). When debriefing, members are addressed by their numbers, as in, "Number Two, what was your thinking when you did..." The point being that a lower ranking member can question and analyze anyone else's performance without fear of retribution. Murphy states that the rankless culture that now exists in the Air Force debriefings helps team members bring up their own shortcomings on a particular mission before someone else does. The fact that this culture does not necessarily exist in the business world means that we must develop it.

[Charles Martin]

It is a great idea and a process that all projects should undergo. After all, the point of experience is to gain knowledge for future experiences.

It is key, though, to have 100% buy-in by all participants. Sometimes a particular employee (especially a manager/supervisor) may say they agree with the "rankless" status, but afterwards, harbor grudges based on what was said by others. Retribution is very often a result of hearing someone else's (usually negative) opinion. This type of attitude is not 100% buy-in because this person does not agree with the idea that he/she needs to improve and/or performed poorly in the eyes of their peers (or underlings).

Also, if you decide that this type of debriefing should be done, then do it when you say you will. I had a boss (same software company I've mentioned before) who promised after the latest "fiasco" that a meeting to air out the problems would be held. Then it was postponed due to another project that was taking up everyone's time (this boss loved to overlap projects and deadlines).

Six months later, we're still being promised that the meeting will happen. Well, it's been two years (and for me, two jobs ago) and I know for a fact that meeting has never happened and no further meetings of this nature have ever been promised again.

[Bert]

Agree, agree, agree!

The 100% buy-in is important and is possible if the rankless and no retribution ground rules are followed. Murphy talked about this in his book as Generals, who are sometimes on flight teams, outrank all other team members but are sometimes the lowest performing member on the team. It is strict adherence to the rankless/no retribution rules that allow free dialog. This is a culture that would have to be developed in a business setting, as there is often a cover-your-butt culture that is often cut-throat in nature already in place.

Yes, a manager's credibility is shot if he/she doesn't follow-through. This is a problem that has derailed many careers.