Onboarding & Evaluation Process

The chart below describes the stages of onboarding and the various evaluation points that occur during the process. Since newcomers may start at any time during the sprint, this process overlays the sprint process. Newcomers are expected to participate in sprint planning meetings, commit to tasks for the upcoming sprint, and practice time management using the sprint planning tools and by updating the Remaining Time estimate fields on their tasks.

As with all things at OpenCraft, this process is continually being reviewed and improved, so please provide any suggestions or feedback on your onboarding task.

Newcomer Weeks
Week -1 Prior to your arrival, we will arrange for a core team member to be your mentor and to review your onboarding task.
We'll also arrange your accounts and access to email, JIRA and the other communication tools.
Week 0 Work on your onboarding task, which involves reading documentation, completing the onboarding course, and setting up an Open edX devstack.
You'll also have a newcomer-friendly task assigned to work on in the first week, after finishing your onboarding.
Attend the 121 meeting scheduled by the reviewer of your onboarding task to say hello and discuss your progress.
If your devstack gives you trouble, be sure to ask your reviewer or on the Mattermost #devstack channel for help, and/or arrange a synchronous meeting to work through any issues.
Week 1 You've likely finished the onboarding course and your devstack setup, and are ready to work on a newcomer-friendly or other small task.
Reach out to your mentor or the sprint firefighter to help find tasks and a reviewer from the core team to help you.
To avoid spillover, we recommend against pulling new tasks into the current sprint in the first instance -- the review cycles can often take more time than expected. So instead, especially if a new sprint is starting soon, commit to a task in the next sprint, and work ahead.
Week 2 At the end of this week, your mentor and 2 other core team members will complete a screening review of your work so far.
This review exists to provide early feedback, and to identify extreme issues like a failure to communicate within 48h of pings on tickets and Mattermost, or cases where excessive time has been logged to tasks without sufficient explanation or outcomes. In this case, we would give notice that the trial period will end. But if you're communicating on your tasks and making progress, then your trial will continue as scheduled. Your mentor will pass on any feedback -- positive and negative -- from this review.
Week 3 By the end of this week, you should have completed some tasks, with story points totalling around 8-12 points. If you haven't, bring this up as soon as possible with your mentor.
If you've had spillover, consider what went wrong during these tasks and talk about it with your mentor.
Take care not to overcommit during the next sprints to get this under control. Time management is one of the hardest parts, so after each sprint ends, take care to ensure that the Sprint Commitments spreadsheet is accurate, and your spillover is improving as you progress through the trial period.
Week 4 By this time, depending on when you started, you've completed 2-3 sprints, so it's time to ensure that you're completing a breadth of tasks to showcase your skills.
Have you taken on increasingly difficult tasks?
Have you submitted a PR to the Open edX platform?
Have you launched appservers or contributed to Ocim?
Have you completed any devops tasks?
Have you been the primary reviewer on some tasks?
If not, try to find tasks for the next sprints which would fill these gaps, and discuss any cell-specific expectations with your mentor.
Week 7 This week will be your developer review.
All the core team members in your cell will review your tasks, PRs, and communications, and vote on whether to accept you into the core team, extend your trial period, or end your trial.
Everyone in the cell has to agree to confirm a new core member. We each do our own evaluation independently, and then discuss if there's a difference of opinion.
Week 8 This marks the end of your initial trial period -- Xavier will meet with you to discuss the results of the developer review.
If you're joining the core team now, congratulations! There will be a small core team onboarding task to complete in your next sprint, and you can continue logging "onboarding" time to your onboarding ticket for a while.
If your trial period has been extended, that's great too! Xavier will provide specific details on the improvements required during the extension, and it's really important to focus on these areas during your extension.
Week 11 If your trial period was extended, the core team will do another developer review, focusing on your improvements during the last 2 sprints.
Week 12 This week marks the end of your extended trial period, if applicable. Xavier will let you know the results of the second developer review.

Evaluation criteria

The screening and developer reviews will be evaluated on the following criteria:

  • Technical skills.
    Team members must demonstrate development and devops abilities on basic and complex tasks.
  • Time management and spillovers.
    Newcomers must have at least 2/4 clean sprints during their initial trial, or 4/6 for extended trials. Sprint status is documented on the Sprint Commitments spreadsheet (linked from each cell's weekly sprint meeting).
  • Communication.
    See Roles: Communication for the expected response times, and the additional expectations for Newcomers.
  • Adaptability.
    Team members should respond gracefully to changes in task requirements and scope, communicate concerns and issues, and allocate effort appropriately across the current or follow-up tasks.
  • Potential for growth.
    Team members should demonstrate an enthusiasim for learning and improvement across all aspects of their work.

Here is some more detail about things the core team look for when evaluating newcomers:

  • Delivering On-Time: Avoiding spillover and delivering on schedule is really important in an environment where we make direct promises to clients about deliverables. Our reputation as an organization is on the line when we cannot deliver as we promised, so it matters tremendously to us to see a newcomer making deadlines consistently. It's required that you communicate explicitly when you feel there is going to be spillover, as soon as you can detect it, and try to find someone else who can complete or help you complete them. It’s totally ok to do this, and even welcomed by people who have time left in their sprint. We are a team, and we work together to avoid spillover.
  • Communication: As stressed above, as an international remote team, there is little progress we can make if we don't constantly communicate (with respect to not being interruptive if it isn't necessarily urgent). We promise you that we didn't recruit any mind readers! We won't magically figure anything out unless it's been talked about, through any of our multiple modes of communication. You should be communicating with your reviewers daily or every 2 days minimum on what your progress on their task is (by commenting on the JIRA tickets). Even if they have no questions, just stating status is important and can give reviewers/mentors somewhere to jump in and help. On the other hand, when blocked in a task, make sure to reach the reviewer for help. If the reviewer isn't available, you can reach for the sprint firefighters.
  • Show your skills: It's important to take tasks of progressive difficulty, take reviews on too. It's much easier for the core team to review your trial if you have picked varied tasks of different complexity and skillset. We’re looking for a cross-section of tasks across all our required work areas: full stack dev, devops, and ops.
  • "Nice": This point is in quotes because everyone obviously likes being around other nice people, so you'd assume this was obvious. But of course everyone believes, "Yeah, I'm nice!", but it goes a long way to being deliberately nice with your colleagues, and not just believing you are; they will simply enjoy working with you more.

Other references

See also: