Analysis of Pair Programming

From Square Root

Given that the team has been expending 48 hours on Studio pairing up on tasks, it is feasible to quantitiatively analyze the effectiveness of pair-programming

Contents 1 Hypotheses 2 Questions 3 Metrics and Data 4 Procedure 4.1 Milestone Category Types 4.2 Defect categories, types, and reasons 4.3 Milestone Allocation 5 Analysis and Reflection 6 References

Hypotheses

1. Pair programming produces code of same or better quality than as Fagan inspection programming alone.
2. Pair Programming requires 15 - 80% more effort to complete a feature than programming alone.
3. Pair Programming requires 60 – 80% less calendar time to complete a feature than programming alone.
4. The most effective pairs have similar problem solving styles, creative problem solving styles, and personalities.
   • Define “effective”

Questions

1. Pair programming produces code of same or better quality than as Fagan inspection programming alone.
   • What is the defect density for code that is passed through Fagan inspection? How does it compare with the defect density for code produced via pair programming?
   • What are the type of defects detected through Fagan inspection? How does it compare with pair programming?
2. Pair Programming requires 15 - 80% more effort to complete a feature than programming alone.
   • What % of time is spent on refactoring existing code that is functional?
   • What % of LOC was refactored in a feature?
3. Pair Programming requires 60 – 80% less calendar time to complete a feature than programming alone.
   • Determine the effectiveness of pair programming in comparison to individual contribution on tasks
   • What is the % increase/decrease in time for completion of a task of ‘X’ units/difficulty?
   • What is the % increase/decrease in quality of the artifact being constructed?
   • How well does pair programming scale to the size of the task being tackled?
   • How well does pairing fare to the nature of the task being tackled?
4. The most effective pairs have similar problem solving styles, creative problem solving styles, and personalities.<DIFFICULT TO MEASURE OBEJCTIVELY>
   • What is the % increase/decrease in time for completion of a task of ‘X’ units by two specific members?
   • Does performance vary based on compatibility, knowledge-base or other factors?
   • Which styles are compatible? Which are not? Why?

Metrics and Data

1. What is the defect density for code that is passed through Fagan inspection? How does it compare with the defect density for code produced via pair programming?
   • Defects/KLOC
   • Potential Defects/KLOC (for pair programming)
   • Defects/Use Case points
2. What are the type of defects detected through Fagan inspection? How does it compare with pair programming?
   • Type of defect/KLOC
   • Type of defect(potential - caught by copilot)/KLOC (for pair programming)
   • Type of defect/Use Case points
3. What % of time is spent on refactoring existing code that is functional?
   • Time spent on refactoring/Total time spent in an iteration
4. What % of LOC was refactored in a feature?
   • LOC refactored/Total KLOC for feature
5. What is the % increase/decrease in time for completion of a task of ‘X’ units/difficulty?
   • Hours spent by pair on a Task of 'X' units/Hours spent by an individual on Task 'X'
6. What is the % increase/decrease in quality of the artifact being constructed?
   • Defects in post development/KLOC (pairing vs individual)
7. How well does pair programming scale to the size of the task being tackled?
   • Defects detected/KLOC over vaious size tasks

Procedure

1. Divide the milestones for iterations 8, 9, and 10 into two categories: Paired or Individual w/ Fagan. The two categories should have approximately the same number of Use Case points.
2. Plan each iteration as we have for iterations 5, 6, and 7. We'll continue to follow the task ownership convention where an individual will own a group of tasks that are roughly related to one another. For example, all or most of the UI tasks for a given milestone.
3. Perform the work according to the task category.
4. All milestones will go through Acceptance Testing once the developers deem the feature as ready. Issues discovered during testing will be recorded in the bug tracker. Additionally, issues discovered once the features are released to the client will also be recorded.

Milestone Category Types

Paired Milestones

• At least 70% of coding and design tasks must be paired. Testing, prototyping, requirements clarification, and other non-coding tasks are not required to be paired.
• All coding and design tasks should have a copilot role associated with the task.
• Task data for each person will be recorded in the task tracker.
• Real time defect data will be recorded and turned in to the tracker following the conclusion of each task.

Individual Milestones

• All work must be done individually. Team members may ask for help from other teammates, but they should not be planned pairing sessions and help should be limited. If a team member does not know how to do something on their own and is struggling with a task, the person should record the time they spent on it, close the task, and create a new task for someone else to take to finish the job. As a rule of thumb, anything more than a few minutes working with someone in front of the screen, working together no longer qualifies as individual work.
• There is no copilot role associated with tasks in this milestone.
• Each milestone must undergo a single Fagan Inspection, inspecting 200-300 lines of code. During the inspection, all aspects of quality will be addressed including coding standards, design, improvements, errors, and bugs. We will follow a strict Fagan Inspection with only four inspectors.

Refactoring Tasks

• Refactoring tasks are separate from new development and there is a separate discipline field in the task tracker (cleverly called "refactoring"). Refactoring tasks should be marked with that discipline.
• Refactoring tasks are tasks where code that is functional but not of optimal standards (as determined by expert judgment) is retouched to improve some aspect of the code.
• The main reasons we refactor are to improve maintainability or performance. We'll try to measure refactoring effectiveness using LOC counters and cyclomatic complexity analysis.
• If you do refactoring, adhere to the milestone category and record it separately from new development.

Defect categories, types, and reasons

We will use the Personal Software Process defect categories, types, and reasons. We chose PSP because we were familiar with it and PSP has a solid process for defect data collection and analysis.

While pairing, individuals will record issues in "real time" using the same categories as the bug tracker. We created a worksheet which is to be passed back and forth among the pair, trading the keyboard/mouse for the paper/pencil. pair-programming-data-sheet.docx.

We will use the data recorded in the bug tracker to determine the quality of milestones. We also want to know how many issues are discovered through our testing and how many are discovered after release.

Milestone Allocation

We want to have an even number of individual and paired milestones in each iteration to reduce risk and, in the end, we want about the same number of points in each bucket. With this in mind, the assignments may need to move around a little.

Use Case	Points	Experiment Assingment
Agree on Definitions	5	Pair
Collect Artifacts	5	Pair
Reset Password	5	Individual
Edit Profile	5	Pair
View User Profile	5	Individual
Create Project	5	Pair
Delete Project	5	Individual
Create or Edit Inspection Technique	5	Individual
Create or Edit Elicitation Technique	5	Mixed
Teach Step	5	Mixed
Close Step	5	Pair
Choose Step	5	Individual
Choose Project	5	Individual
Create/Edit Categories	5	Mixed
User Manual and other Artifacts	5
Square Lite	5
Identify Assets and Goals	9	Mixed
Prioritize Requirements	9	Pair
Perform Security Risk Assessment	9	Mixed
Select Security Elicitation Technique	9	Mixed
Elicit Security Requirements	9	Mixed
Register New User	9	Pair
Install Square Tool	9	Individual
Export Data	9	Pair
Manage Site	9	Mixed
Log In/Log Out	10	Mixed
Inspect Requirements	14	Individual
Privacy Questionaire	14
Categorize Requirements	14	Mixed
Manage Project	14	Mixed
Integration Proof of Concept	14

Analysis and Reflection

References