As a Product Owner, the scrum framework will significantly help you manage your project or product. It can help you and your team to get a better overview, realize results faster, have a better time estimation and create greater motivation within the team members.

What was Scrum again?

The process model of agile software development assumes that software projects cannot be planned in detail in advance due to their complexity. For this reason, planning takes place according to the principle of step-by-step refinement, with the team developing the system on an almost equal footing. The team can consist of Developers, Data Engineers or Data Scientists.

Roles in a Scrum Data Science Team — Image Source: Adi Wijaya — Scrum in Data Science, What Works and What Doesn’t

The Product Owner is the link between the customer and the team and prioritizes the user stories, for example. In addition, there is a Scrum Master who uses good soft skills to keep the methodology and the team spirit in check [1].

Scrum Ceremonies

Talking about stories and tasks it is important to describe them very well — especially in the field of Big Data. Mostly technical details have to be documented well so errors and questions from developers can be kept low. Here, I recommend a backlog refinement anti-cyclically to the sprint planning, where the product owner and the team can discuss stories and how to implement them technically.

Thinkable Agile Process — Image by Author

Ready and prioritized stories could then be marked and pulled into the next sprint. At the end of the sprint, the team gets together in a review. The product owner can decide whether or not to release the increment. While in the retrospective the team comes together to document and discuss what worked and what didn’t work in a sprint.

Data Science & Scrum

Data Science is a fairly new field and things are constantly changing and new insights are being gained. There is no manual that will be valid for the next twenty years. With Scrum, new insights, methods and technologies flow immediately into the work process.

Jira Sprint — Image by Author

Data Scientists must have the openness to recognize that they will only find out in the course of a project how analyses work and what significance the insights derived from them have for the company. That is why Data Science and the Scrum Framework fit together very well here.

Summary

Since the work of a Data Scientist is similar to that of a normal software project, it is not surprising that the Scrum framework with its roles, ceremonies and methods is also helpful here. If the roles and ceremonies of Scrum are established in the team, together with the methodologies such as user stories, good estimation procedures and a tool that supports the work, a powerful Data Science team can be created. Which of course doesn’t mean that you have to explicitly follow everything Scrum says, most companies rather adapt the framework to their needs.

Sources and Further Readings

[1] Claire Drumond, What is Scrum? (2021)