云效研发效能度量体系，如何展示和解读交付效能数据

Cloud effectiveness R & D effectiveness measurement system, how to display and interpret the delivery R & D performance data, at the end of an iteration or a cycle, the team needs to review and drive R & D performance improvement, and needs to show the team’s current R & D performance data before the review. Through the R & D performance measurement to measure whether the team has the ability to deliver value.

Alibaba’s R & D performance measurement system (as shown below) measures whether the team has the ability to deliver value with “smooth and high quality” through five dimensions. including demand response cycle, continuous release capability, delivery throughput, delivery process quality and delivery quality, looking forward to fast, more and better delivery requirements.

Specifically, it contains two detailed indicators, which are:

Delivery cycle: refers to the average time it takes from the confirmation of the requirements put forward by the user to the launch of the requirements. It reflects the responsiveness of the team (including business, development, operations, etc.) to customer issues or business opportunities

Development cycle: refers to the average time it takes from the time the development team understands the requirements to the time the requirements can be brought online. It reflects the responsiveness of the technical team.

The distinction between the delivery cycle and the development cycle is to decouple and identify problems in order to make targeted improvements. Among them, the delivery cycle is the ultimate goal and test standard.

Cloud effect has rich report statistics functions. Next, I will take you to learn how to configure the above report on cloud effect.

As shown in the following figure, enter from “Statistics” and click “New report” to see the list of new reports for cloud effect.

Select performance Analysis, and then click demand Cumulative flow Graph to appear the report shown in the following figure:

Each blue dot represents a published requirement, the horizontal axis is the date, and the vertical axis is the number of days. The more down these blue dots are, the better, which means that the shorter the delivery time of the demand is, the better the response is; the higher the density of the scatter is, the higher the delivery frequency is, and the more uniform the horizontal distribution of the scatter is, the better the stability of continuous delivery.

Select performance Analysis, and then click delivery cycle report

As shown in the following figure, select monthly statistics, select “requirements” for the task, “selected” for the start status, “released” for the end status, and save after updating the chart name of “delivery cycle trend Chart” in the upper left corner.

Select performance Analysis, and then click delivery cycle report

As shown in the following figure, select weekly statistics, select “requirements” for tasks, “ready (to be developed)” for the start status, “to be released” for the end status, and save after updating the name of the chart in the upper left corner to “Development cycle trend Chart”.

The following figure is an example of a development cycle trend chart. The horizontal axis is the time, arranged according to the natural cycle, and the vertical axis is the number of requirements. Each column represents the number of requirements to be released in the current week. Each column is made up of different colors. Blue represents the development cycle of the requirement is less than 1 week, green represents the development cycle of the requirement is between 1-2 weeks, orange represents the development cycle of the requirement is between 2-4 weeks, and red represents the development cycle of the requirement is greater than 4 weeks.

As can be seen from the above figure, the number of requirements in the requirements development cycle continues to grow within one week, and the share within one week is also gradually increasing. Because the data was taken on August 15, the data for the last week is not complete.

Specifically, it includes two subdivision indicators, which are:

Release frequency: the speed of the team’s external response will not be greater than its delivery frequency, which restricts the flow speed of the team’s external response and value. It is measured by the number of valid releases per unit time.

Release lead time (also known as change lead time): that is, the time it takes from code submission to feature launch, it reflects the basic capabilities of the team release. If you spend a lot of time, it is not appropriate to increase the frequency of distribution.

The change of continuous release ability can be seen from the demand control chart. if the density of the demand scatter is higher, the delivery frequency is higher, and vice versa.

The release lead time has a lot to do with the team’s engineering capability, which is strongly related to the cloud effect new product code platform and pipeline, which will not be explained here.

Refers to the amount of demand delivered per unit time. On this point, the common question is, does the number accurately reflect the efficiency of delivery? This is a problem. Therefore, we put more emphasis on the comparison of the requirements throughput of a single team, which is statistically sufficient to reflect trends and problems.

The trend of demand throughput has also been shown in the chart of the demand response cycle, and the vertical axis in the following figure represents the number of requirements released this week, so the higher the column, the more requirements are delivered this week.

Here, the throughput is calculated according to the number of requirements. When the granularity of the requirements varies greatly, the throughput data will deviate, so before counting this, the team is expected to split the requirements into the workload within 2 weeks.

Select by iteration and task status in the custom chart, and then add two filters: iteration and status. Iterations select several iterations that need to be compared, and the status selects only published requirements. The throughput trend chart shown below is compared by iteration.

It contains two subdivided indicators, which are:

Time distribution of defect creation and repair during development: we hope that defects can be found continuously and in a timely manner, and repaired as soon as possible.

Defect inventory: we want to control the defect inventory throughout the development process, keep the product close to being releasable, and lay the foundation for continuous delivery.

The core of delivery process quality is built-in quality, that is, the quality of the whole process and the whole period of time. Instead of relying on a specific phase, such as the test phase, or a specific period of time, such as the later stages of the project. Built-in quality is the basis of continuous delivery, and detailed examples of its specific measurement methods are given below.

As shown above, the Abscissa in the figure is the date, and the red vertical bar above the Abscissa represents the number of defects found on that day; the green vertical bar below the Abscissa represents the number of defects solved that day; and the orange curve represents the stock of defects. The left and right parts of the figure compare the two delivery models.

In the left half, the team belongs to the development mode of small waterfall. In the early “iteration”, the team focused on design, coding, and introducing defects, but not immediately integrated and verified. Defects were hidden in the system until the later stage of the project, when the team began to integrate and test, and defects broke out.

In the small waterfall mode, the process quality is poor, resulting in a large number of rework, delays and delivery quality problems. In this model, the delivery time of the product depends on when the defects can be fully removed, of course, can not achieve continuous delivery, nor can it quickly respond to external needs and changes. Moreover, this model usually leads to the rush of work in the later stage, burying the hidden trouble of delivery quality.

In the right half, the team began to evolve towards a continuous delivery model. Throughout the iteration, the team develops small-grained requirements, continuously integrates and tests them, and instantly discovers and solves problems. The defect inventory is controlled and the system is always close to being releasable. This model is closer to the continuous release state, and the team’s external response ability is enhanced.

The defect trend map reflects the team’s development and delivery model from one aspect. It guides the team to identify defects continuously and early and remove them in a timely manner. Control defect inventory, so that the system is always close to the releasable state, ensuring the ability of continuous delivery and external response.

In the project Statistics, select defect Analysis, and then click defect trend to appear as shown in the following figure.

It contains two subdivided indicators, which are:

Together, the two determine the availability of the system.

Alibaba R & D effectiveness measurement system measures whether the team has the ability to deliver value smoothly and with high quality through five dimensions. Through the above five groups of indicators, this paper tells a complete story from the three aspects of mobility efficiency, resource efficiency and quality, and answers the core question of the current organization’s ability to deliver value continuously. Among them, continuous release capability and demand response cycle reflect the flow efficiency of value; throughput reflects resource efficiency; and delivery process quality and external delivery quality reflect the quality level together.

After reading it, don’t forget to like, collect and follow you.