Skip to content

Epic Planning and Sustainability Management

Epic Planning

The Lifecycle of an Epic

  1. Most epics start with a discovery based on a client requirement. (For internal projects, the client is OpenCraft).
  2. An epic is created based on the corresponding discovery and starts in the Prospect column.
  3. The discovery and corresponding estimates are shared with the client, and the epic is moved to Offer / Waiting on client.
  4. If the client accepts the work, the epic is moved to Accepted.
  5. Once actual development starts, the epic is moved to In Development.
  6. Once the client's requirements are met, and the client has access to the completed work, the epic can be moved to Delivered to client / QA.
  7. The epic should be moved back from Delivered to client/ QA to In Development if the client requests additional work or modifications that need development.
  8. When all the work in the epic is complete (for instance if all upstream PRs have been reviewed and merged) the epic can be moved to Done.

Recurring epics are generally not based on a project or discovery, but are used to track work for different cell roles (or long-running maintenance contracts for specific clients).

Tracking Epic Status

  • Each cell has its own Epics board and epics, and is responsible for making sure that the projects represented by the epics are being properly delivered based on the lifecycle described above.
  • To find the the epics board for your cell, go to Boards > View all boards > Epics - <your cell> in JIRA. (Note that if you've recently accessed the board, it may also be listed under Boards > Recent boards.)
  • Every sprint, status changes of individual epics from the past sprint should be reviewed and evaluated to be able to keep both the Epics board and the epic planning spreadsheet up-to-date. This involves making sure that:
    • Each epic that is In Development has an epic update.
    • New epics are moved through the set of initial states (Prospect, Offer / Waiting on client, Accepted) as appropriate until they reach In Development.
    • Epics that have become blocked on the client are moved to Offer / Waiting on client.
    • Delivered epics are moved to Delivered to client / QA.
    • Completed epics are moved to Done.
  • Every sprint, the epic planning manager should compare the Epics board for their cell with the corresponding sheet(s) of the epic planning spreadsheet and update the spreadsheet as necessary. This involves:
    • Adding new clients and/or epics.
    • Updating info about client and/or epic ownership.
    • Removing entries for clients and/or epics from future months based on contract and/or completion status.

Recording High-Level Information about Epics

We keep a record of all epics that we work on (except onboarding epics) in the Time / Estimates sheet of the epic planning spreadsheet so that high-level info (such as who estimated and led an epic, and how long it took to complete) can be referenced as necessary in the context of future discoveries.

As part of the epic planning process, the epic planning manager is responsible for keeping the Time / Estimates sheet up-to-date by:

  • Adding entries for new epics.
  • Updating entries for completed epics.

Tracking Cell Member Availability

To be able to determine capacity allocations for individual projects we need to track cell member availability. As an epic planning manager you should:

  • Keep a look out for newcomers and core members joining or leaving the team, and add their details to the epic planning spreadsheet.
    • For newcomers this includes adding onboarding time to the onboarding section and specifying their availability in the availability section of the epic planning spreadsheet. Only make these updates when the corresponding information is fully public, i.e., the newcomer should know whether they have been accepted or not by the time these updates are made.
    • Availability for each member is calculated in full-time weeks. For example, if someone's commitment is:
      • 40h/week, their availability will be 40 / 40 = 1.
      • 30h/week, their availability will be 30 / 40 = 0.75.
    • If you prefer you can also specify availability of individual cell members in hours per week. Just note that if you do, you'll also need to modify the calculation of total availability accordingly. (I.e., instead of just summing up the availability of the cell members to get the number of person-months available for each month, you'll need to sum up the weekly hours and divide the sum by 40.)
    • For team members leaving the team, make sure to remove (or reset to zero) their availability for the remaining months of the year.
  • Handle requests from cell members to change their weekly hour commitments, if the requested commitment is in the range of 30-40h per week:
    • Outside of that range, refer to the CEO: We generally avoid commitments that are lower or higher than 30-40h/week because they may
      • prevent cell members from taking some types of tasks,
      • increase overhead for commitments below 30h/week, and
      • can lead to burnout for commitments that exceed 40h/week.
    • However, temporary changes lasting 1-2 months that fall outside of the 30-40h/week range are generally acceptable.
    • If you need time to implement the change, for example to finish a round of recruitment, don't hesitate to discuss a specific date at which to make the change. Generally we always agree to requests for changing weekly hour commitments, but not necessarily to implementing them immediately.
    • To implement the requested changes:
      • Update the cell member's availability in the epic planning spreadsheet to take the impact on your cell's total availability into account for epic planning.
      • Update the cell member's hours by via Tempo > Administration > Workload schemes:
        • Locate the team member's current workload scheme (e.g. 35h), and click Members.
        • Click Move to move the team member to the new workload scheme (e.g. 30h).
  • Check the calendar for vacations coming up in the next couple months:
    • If someone has scheduled time off for a total of 1 week or longer, comment on the cell(s) in the epic planning spreadsheet that represent their availability for the month(s) during which they will be away:
      • Mention the number of weeks they will be away.
      • Mention the remaining availability for the affected month(s).
    • Don't change their availability for the affected month(s) in the spreadsheet: Holidays are already taken into account in the holidays buffer, which reduces the cell's monthly availability according to the cap on the number of cell members that can be off at the same time. (That cap is usually 20%.)

Capacity Requirements

  • For each client and epic, maintain a count of the amount of time required to complete the accepted project scope over the next months in the epic planning spreadsheet for your cell. This is used to inform recruitment needs (cf. below). Note that the amount of time required to complete the accepted scope of a project isn't always equal to the estimated budget for the project:
    • Work may progress more quickly than expected in some cases, allowing us to reduce capacity allocations for coming months.
    • In other cases there might be a need for additional work that was not accounted for in the original estimates, requiring us to increase capacity allocations for coming months.
  • Ensure delivery and bugfix deadlines for individual epics are on target. If that's not the case, comment on the epic(s) or directly on the epic planning spreadsheet, pinging epic owners as necessary and reminding them to:
    • Determine the impact of shifting deadlines and figure out how to minimize it.
    • Discuss the situation with the client (or the CEO in the case of accelerated epics), negotiate scope adjustments as needed, and make sure that the client (or CEO) is OK with them.
  • Coordinate with your cell's recruitment manager to slow down/speed up recruitment as necessary.

Budgeting

  • TODO: Info about adding actuals to the epic planning spreadsheet.

Working with SprintCraft

Creating budget entries
  • To tell SprintCraft about budgets for individual accounts, set them via Django Admin > Sustainability > Budgets.
  • In JIRA, accounts are identified by a name (e.g. "Self-hosted instances maintenance") and a key (e.g. "SELFHOSTED-MAINTENANCE").
  • Account names may have a prefix (such as "(Inc) - ") which specifies the legal entity through which the corresponding work logs will be invoiced.
  • To make it possible for SprintCraft to correctly associate a budget entry with a JIRA account, the Name field of the budget entry needs to be set to the account's name.
    • Note that even though SprintCraft does not display prefixes of account names on the Budget dashboard, they must be included in the Name field when creating budget entries.
    • Note that on the rare occasion an account is renamed in JIRA, budget entries for that account will need to be updated accordingly in SprintCraft. Otherwise SprintCraft will report a 0h-budget for the affected account.
  • If the budget for a specific account is the same for multiple months, there is no need to create a budget entry for each month. Just create a single entry for the first month to which the budget applies; SprintCraft will automatically apply it to the following months.
    • Note that this means you must explicitly reset the budget for a specific account to 0h when the corresponding contract ends (or when we've run out of budget for the account). Otherwise the budget will be applied to future months indefinitely.
  • If you create multiple budget entries for the same month (e.g. Jun 2021), SprintCraft will use the newest budget entry for display on the Budget dashboard and for sustainability calculations. (In other words, it is possible to overwrite earlier budget entries for a given month and account by creating new budget entries for the same month and account later on).
  • Budgets are specified in (full) hours per month. (Decimals are not accepted.)
Choosing appropriate budgets
  • To get accurate numbers for quarterly sustainability reviews, you'll need to create appropriate budget entries for all client accounts belonging to your cell.
    • A client account belongs to your cell if the client owner is a member of your cell.
    • On the Budget dashboard, client accounts have a category of "Billable" and are listed at the top (above non-billable non-cell and non-billable cell accounts).
  • Budget entries should reflect reality: For each month since the beginning of the (first) contract belonging to a given account, budget entries should match the number of billable hours that we had available for that month.
    • ⚡ If the sum of all budget entries for a given period exceeds the number of hours that we were able to bill for that period, this will artificially lower your cell's sustainability ratio and hide budget issues.
    • ⚡ If the sum of all budget entries for a given period is lower than the number of hours that we were able to bill for that period, this will artificially increase your cell's sustainability ratio and inflate budget issues (i.e., it will create the illusion of budget excess even if there wasn't any / even if it was lower that the sustainability numbers suggest).
    • When in doubt, check with the administrative specialist to get more information about the number of hours that were billed to a client for a specific period.

Keep in mind that budgets are not the same as capacity allocations: When a project first starts, we usually base capacity allocations for coming months on budgets. However, there are several situations where budgets and capacity allocations may no longer be tied together as closely as they were in the beginning of a project. For example:

  • We may have more budget left than hours needed to complete the scope of an epic in the coming months.
  • An epic may have a lot of budget left but the work may become completely blocked on the client, upstream etc. (which means we can lower capacity allocations for the coming months to 0h).
  • An epic may run out of budget before the agreed-upon scope has been completed.
Reviewing budget information
  • To review budget information for one or more accounts in SprintCraft:
    • Go to the Sustainability dashboard for your cell (e.g. Falcon).
    • Select the desired period to review (e.g. Jun 1st 2021–Dec 31st 2021).
    • Hover over the name of a specific account with your mouse to get a list of budgets entries for:
      • individual months of the current year, starting with the first budget entry that was explicitly set for that year in SprintCraft.
      • the first month of the next year.
    • For high-level information about account budgets check the Period Goal / YTD Goal and Period Spent / YTD Spent columns. For each account:
      • The Period Goal column shows the total number of hours budgeted for the selected period (i.e., the sum of all budget entries for the selected period).
      • The YTD Goal column shows the total number of hours budgeted from the beginning of the first year within the selected period (i.e., Jan 1st in the example mentioned above) to the end of the next sprint.
      • The Period Spent column shows time logged during the selected period.
      • The YTD Spent column shows time logged from the beginning of the first year within the selected period.
    • At the bottom of the Budget dashboard you'll see an item called "Overhead". This item does not correspond to an account. Instead, it lists computed values for YTD Spent and Period Spent that represent the sum of non-billable hours logged against client accounts for the corresponding time spans (i.e., YTD and selected period).
  • If you find any inconsistencies or outdated information, adjust budgets in the Django admin as necessary (as described above).
  • Reload SprintCraft to see your changes reflected in popups and YTD Goal / Period Goal columns.

Budgeting workflows for different types of epics

Fixed scope / Fixed budget
  • Initially:
    • Spread budget out over coming months based on delivery deadline and capacity allocations for the epic. Budget entries for individual months should sum up to total budget from quote/contract.
  • While project is running:
    • Adjust budget entries for past months (or periods) as necessary: If actuals for past months differ significantly from budgets that were set initially, budget entries will need to be adjusted to match actuals more closely. (This makes it possible to get accurate numbers for quarterly sustainability reviews while the project is still running.)
    • If the project runs out of budget before its scope is complete:
      • Adjust budgets for past months to match actuals.
      • For the month during which the budget ran out, set the budget to the remaining hours for that month (i.e., the number of hours that was left in the budget at the end of the previous month).
      • Set the budget to 0h from the following month on.
  • When project is over:
    • If no budget left, or project went over budget (same as above):
      • Adjust budgets for past months to match actuals.
      • For the month during which the budget ran out, set the budget to the remaining hours for that month (i.e., the number of hours that was left in the budget at the end of the previous month).
      • Set the budget to 0h from the following month on.
    • If below budget: No changes needed (unless some budget updates were missed while the project was running, i.e., unless budgets and actuals still differ significantly for one or more past months).
    • Either way: Make sure that budget entries for individual months sum up to the total number of hours that the client paid for (i.e., the total number of hours from the initial quote/contract for the project, plus any budget extensions that the client approved in the meantime).

Note that the current implementation of SprintCraft does not take unused hours from fixed-price epics (which we sometimes refer to as "anti-overhead") into account for sustainability calculations. More specifically, when calculating "Overhead", SprintCraft does not subtract the sum of unused billable hours from fixed-price epics from the sum of non-billable hours that other client epics may have incurred.

Dynamic scope / Monthly budget
  • Initially:
    • Set budget for coming months to the number of billable hours that we have available per month.
    • If the end date of the current contract is known, set the budget for the month that follows the end date to 0h.
  • While project is running:
    • Adjust budget entries for past months (or periods) as necessary: If actuals for past months differ significantly from budgets that were set initially, it may make sense to adjust budget entries to match actuals more closely. (As explained above, the goal is to always get the most accurate numbers possible for quarterly sustainability reviews.) Any adjustments you make should be based on the following criteria:
      • If actuals for past months are higher than budgets (as currently defined):
        • Increase budgets to match actuals if the situation was dealt with via a one-time budget extension (i.e., if the client agreed to pay for all hours logged in excess of the default monthly budget).
        • Increase budgets to match new monthly budget if the situation was dealt with via a permanent budget extension (i.e., if the client agreed to increase the default monthly budget to a higher number of hours). Only apply the increase from the first month for which the budget extension was approved.
        • Increase budgets to match actuals if the overhead incurred for a specific month was offset by coming in below the default monthly budget in later months.
      • If actuals for past months are lower than budgets (as currently defined):
        • Lower budgets if:
          • The client is billed hourly (i.e., the monthly budget is not a fixed number of hours that the client pays for each month, irrespective of how many hours we actually spend).
          • Working fewer hours was a deliberate decision to offset overhead incurred in previous months.
    • If the monthly volume of work drops or increases significantly, the epic owner may work with the client to come up with a new monthly budget cap. (You should be able to pull information about these types of changes from epic updates.) Adjust or set budget entries for future months to match the new budget cap.
  • When project is over:
    • Make sure budgets for previous months match billed hours. (This should already be the case if budgets were updated while project was running, but is still worth double-checking at the end of a project.)

Note that you may encounter some situations and/or edge cases that the instructions above don't explicitly cover. In these cases the main aspect to guide your decisions should be that budgets set in SprintCraft should never exceed the number of billable hours that we had available in the past, present, or future (as this would hide sustainability issues).

Dynamic scope / No specific budget
  • Initially:
    • If there is an approximate volume of hours that we are expected to work per month, set budget for coming months to that volume. Otherwise, set the budget to 0h.
  • While project is running:
    • Set budget entries for past months to match actuals. Double-check epic updates for information about non-billable hours incurred in previous months, and take it into account when setting budgets. For example, if the number of hours logged against a specific account last month is 50h, but the client only paid for 40h (for whatever reason), the budget for last month should be set to 40h in SprintCraft.
      • Hours may be non-billable if they exceed agreed-upon budgets for specific tasks: If the client approved a specific budget for a task or set of tasks, we need to stop and ask for a budget extension before spending more time on the task(s). If we ask for retroactive approval, there is a chance that the client won't agree to pay for any/all hours exceeding the original budget, which will make these hours non-billable.
  • When project is over:
    • Make sure budgets for previous months match billed hours. (This should already be the case if budgets were updated while project was running, but is still worth double-checking at the end of a project.)

Sustainability Management

  • Each cell is meant to be a sustainable entity by itself. Its members are the closest to most of the work that impacts its sustainability - the successful estimation and delivery of each client project.
  • Some of the budgets for internal/non-billable accounts are also decentralized to individual cells. See Cell Budgets for more details.
  • The sustainability manager is responsible for ensuring that the cell keeps the budgets it is responsible for in order. This involves the following activities:

Every sprint:

  • Sustainability preview for upcoming sprint.
    • When planning the next sprint, the sustainability manager should:
      • Review the amount of billable and non-billable work scheduled by the cell for the next sprint.
      • Warn epic owners about any budget issues and get them to address those proactively. If the situation cannot be redeemed immediately, the sustainability manager should get a timeline and plan from the concerned epic owner(s) to resolve the issue.
    • The Budget section of the SprintCraft dashboard includes Left this sprint, Next sprint and Remaining for next sprint columns that can be used to estimate next sprint's sustainability.
    • The sustainability preview for a specific cell should be posted on the cell's epic planning and sustainability epic (or integrated into the cell's main epic planning update for the current sprint.) See this section of the roles page for the template to use for the sustainability preview.

Every month:

  • Accounts review for previous month.
    • At the beginning of each month, after the deadline for recording/adjusting time logs for the previous month, the sustainability manager should check that the tasks worked on over the course of the previous month are using the right accounts.
    • For the purpose of efficiency, this review can be combined with adding actuals for the previous month to the epic planning spreadsheet.
    • In particular, incidents that affected one or more client instances should use the clients' Instance Maintenance accounts instead of an internal DevOps account.
    • This review needs to happen before invoices are sent to clients, and results should be posted as a comment on the "Epic planning and sustainability management - <your cell>" epic.

Every quarter:

  • Sustainability review.
    • Every three months the sustainability manager should post an update on the forum about the current status of the cell's sustainability ratio and budgets for internal work, as well as plans for the coming months that ensure sustainable ratios.
    • This is done in January (Q1 update), April (Q2 update), July (Q3 update), and October (Q4 update).
    • It must wait until after client account budgets for the previous quarter are up-to-date to make sure that the reported numbers are correct.
  • Budget updates for non-billable cell accounts.
    • After each sustainability review, the sustainability manager should review and update budgets for non-billable cell accounts in SprintCraft, in coordination with owners of internal epics that are associated with these accounts.