3 business personas in ML, their challenges and opportunities

Published v3 by Nathan Benaich on 6 Sept 2020. (Updated from v2 on 26th March 2020 and v1 on 22nd October 2019).

Introduction

As machine learning (ML) grows in popularity amongst large technology companies and startups, businesses of all kinds are taking notice and asking themselves how they can follow suit. There are many resources that help codify the discovery of real-world use cases where it makes technical and commercial sense to use an ML-based solution. For example, inspired by the popular startup business model canvas, the ML Canvas does a great job at structuring this discovery process.

However, less has been written about the common challenges faced by specific business personas as a function of their level of expertise in ML and the maturity of their data infrastructure. From my vantage point as an investor in AI-first technology and life science companies with Air Street Capital, I will describe three business personas that I have observed as well as their common challenges, technology stacks, talent requirements and organisational structure. The goal here is to help you define your starting position such that you can make informed decisions as the steps in your ML journey.

Three business personas

Broadly speaking, I have observed three different business personas that can be distinguished by the extent to which ML, data infrastructure and software is a core value creator today.

1. The ML-first company

Description: This business persona is creating a product uniquely enabled by ML. For example, this could be an enterprise cybersecurity product that monitors, detects and protects against malicious network activity. The management/senior team are ML natives and have extensive experience with state-of-the-art tools (and often research code) and processes.

Technology choices: Invest significantly into proprietary tooling and adopts open source as much as possible. They will depend on cloud vendors for commodity infrastructure, but will shy away from purchasing third party services that play to the lowest common denominator ML user for problems that they can solve themselves. Examples: Ravelin.

Common challenges: Creating a resilient, reliable, scalable, fast, secure, and cost-effective technology stack. Read more on this topic in an excellent post on data science best practice here.

Talent requirements: Examples include: hiring ML Research Engineers (making R&D models fit for production), Data Engineers (Software Engineers who specialise in building tooling for manipulating large datasets), ML Engineers (training and evaluating models for production use cases), DevOps for ML (or MLOps, pushing and monitoring trained ML models in production), and Data Scientists (folks who write write queries to explore datasets, find relevant signals, and sometimes train models). Roles focused on either a) advancing state of the art in production, b) reliable, large scale ML production system engineering.

Setup of ML organisation: A core ML platform team that is responsible for building a platform to make model building, training, deploying, and monitoring simple and as automated as possible. Then, several product-focused ML engineers or data scientists who are embedded in product teams. These individuals use the core ML platform to train problem-specific models, evaluate them and serve them in production.

2. The software-native company

Description: A software company that is strategically evolving its offerings to be ML-first. This persona might be a workflow SaaS company or enterprise software business whose product essentially consists of a client-side application, server-side application infrastructure, and databases. There is no real use of ML baked into the product when it was built.

Technology choices: This type of company will usually have the necessary business intelligence infrastructure in place (i.e. a data-warehouse) and is looking to deploy internal or hired data scientists in order to leverage their data assets. However, since the existing software architecture was typically built for a different purpose, it might be challenging to deploy models in production. They will often look towards simple to use open source tools, services offered by cloud vendors, or other SaaS companies that provide ML tooling. A focus is placed on customizability, cost, and support. For example, see OpenDoor.

Common challenges: The challenges that the data team will face are around convincing business stakeholders of the value of ML. This persona tends to struggle knowing where and how to begin. If no one in the business has experience with ML, the challenges include defining a low-hanging and addressable problem, deciding to build or buy technology, and determining what success looks like.

Talent requirements: While these businesses have software engineering expertise, they often do not have particular skills in ML. They might have individuals who can prototype a model using a data science notebook or who can integrate cloud available prediction APIs into a functioning demo. Should they hire new people and, if so, should these be data engineers, data scientists, or ML engineers? In many cases, one could argue that these businesses should first hire a data-focussed business person to project manage rather than going directly to hire technical talent. For more, read this piece on setting up analytics competencies from scratch at different stages of company growth.

Setup of ML organisation: For this persona, it often makes most sense to start by pairing a group of software engineers and data scientists with a particular product organisation. The goal here is to scope, build and test new ML prototypes. If this is successful, weigh up the merits of building out an organisation as in the first persona.

3. The incumbent company

Description: This business offers software-based products or uses software to power their business but it really treats software as “old-school IT”. This persona does not see software as a core competence or a vector of differentiation against their competitors. Software is not really a value creator, but more of a means to an end.

Technology choices: This persona will most likely be using many legacy IT systems, will generally be early in their migration to the cloud and does not tend to procure using SaaS or similar low-friction approaches. They tend not to use much open source and instead by software from large IT vendors or have custom solutions written for them.

Common challenges: This persona is likely to want an end-to-end platform of some kind that offers lots of hand holding, starting with business intelligence, analytics and data warehousing. The biggest challenge this persona faces is connecting IT with the business so that “Excel users” can consume data produced by data infrastructure engineers. Here is a great post from dbt that digs into this problem. It is probably too early to be worrying about ML :-)

Talent requirements: Little provided internally. Instead, they will depend on external development agencies or large technology consultancies.

Setup of ML organisation: Pretty much nonexistent.

Bonus section: A list of helpful ML and data tools

The ecosystem of ML and data infrastructure tooling is evolving quickly. To help you bootstrap your search,here is a selected list of solution providers and open source offerings tackling specific parts of the ML value chain:

• End-to-end platforms: DataRobot, Polyaxon, MLflow (open source), Peltarion, Petuum, Domino Data Lab, BigHead (open source), Google’s AI Platform, Spell, MSFT Azure, AWS Sagemaker, Flyodhub, Metaflow (open source), Valohai.

• Model deployment platforms: Kubeflow (open source), Seldon, Algorithmia, Datmo (now part of One Concern), Atalaya, Kubeflow.

• Data science collaboration: Dataiku.

• Notebooks: Deepnote, Neptune, Jupyter, Google's Colab.

• Data pipeline building: Pachyderm, Airflow, Google Dataflow, Amazon step functions, Dagster.

• ETL: Fivetran, Matillion (ETL), Talend.

• Workflow Orchestration: Airflow (open source), Luigi.

• Distributed programming frameworks (batch): Hadoop, Spark.

• Distributed programming frameworks (real time): Spark Streaming, Flink, Kafka streams, .

• Version controlling for models and data: DVC, MissingLink, Verta, Pachyderm.

• Experiment tracking and analysis: Weights and Biases, Comet, MissingLink

• Data labelling: Scale, Mechanical Turk, Heartex, Deepen, LabelBox, Superannotate, Lionbridge.

• Data quality and metrics monitoring: Datakin, Soda Data, Data Gravity, Toro Data, Great Expectations, Hubble, Transform Data.

• Data catalogue: Dataframe, Lyft's Amundsen (open source).

• ML task specific end-to-end platforms: platform.ai (computer vision).

• Trained model marketplaces: Runway.

• Distributed training acceleration: EngineML, Horovod (open source).

• Model monitoring, auditing, explainability, and bias detection: Untangle, Fiddler, Bulletproof.ai, Seldon, Monitor.ml, Mona Labs.

• Feature stores: Tecton, Google's Feast.

For even more depth, check out this compendium on GitHub.

Special thanks to Hendrik Brackmann and Moritz Müller-Freitag for critical review and input on this piece.