Our thoughts on everything related to AI

The need to automate the data labeling process

Machine Learning is a technological wave whose impact on industries and the world can be compared to the emergence of PCs in the 1970s. While the first work now recognized as AI was done in 1943, it wasn’t until the 2010s that AI became widely used in consumer tech products and industries.

There are three types of Machine Learning Models:

1. Supervised Learning Model — where the model needs to be trained in order to perform a specific function.

2. Unsupervised Learning Model — where the model can perform a specific task without any supervision.

3. Reinforcement Learning Model — where the agent learns on its own through positive and negative reinforcements, much like a baby does.

While there have been recent advancements in the field of Reinforcement Learning and Unsupervised Learning, it is Supervised Learning which is the most widely used form of Machine Learning.

With the widespread use of supervised learning models, the need for training datasets is very high in the ML industry. As a result data labeling has become a very important part of the training phase of the machine learning lifecycle.

Data Labelling or Data Annotation is an emerging sector in the world, however, data labeling is primarily still done manually.

The problem?

• With human annotators, the turnaround time for data annotation can be high, usually weeks or even months.
• The labels may be prone to human errors as the data annotator may not have the right domain knowledge.
• The process can be expensive, costing companies thousands of dollars, especially if the volume of the data is high.
• Data privacy may become an issue, especially in cases where the annotation process is being crowdsourced.

So it could take you weeks and cost you hundreds of dollars by the time you get around to training your model.

Now, what if we automated the data labeling process?

Let’s draw a comparison.

Given the above example, while the labels had above 95% accuracy, with manual data labeling it took 4–5 weeks to put Instance Segmentation labels for a 30,000 images dataset.

It cost the clients roughly USD 5000 to get the training dataset ready.

The privacy of the data was at risk because the annotation job had to be crowdsourced to annotate the images within a 4–5 week timeframe.

With our automated data labeling solution, 90% — 95% of the dataset is being labeled by the models. Human annotators assist this process by annotating 5% — 10% of the dataset to make the models domain-specific and by reviewing the model’s label predictions to ensure quality.

With this process:

The dataset was labeled in 1.5 weeks instead of 4–5 weeks because it takes our software a second to label each example.

It cost clients USD 600 to get the training dataset ready because with an automated process the pricing is per example instead of per annotation (as is the case with manual labeling).

100% data privacy was ensured because we used software to label the data. As a result, such large data could be labeled in-house without the need to crowdsource any part of it.

The labeling quality was maintained and the labels were above 95% accurate.

A fully unsupervised model for data annotation isn’t practically feasible, which is the reason why a combination of machine learning and human checks is the best path forward.

This ensures that the labels are as highly accurate as possible. A lot of time is saved by using Machine Learning to get predictions. The software can label each example in seconds. Once the model has given its predictions, the label predictions are reviewed. In the reviewing stage, the annotations that aren’t accurate are re-shaped and don’t need to be annotated from scratch. This minimizes the time taken to review the predictions significantly and in turn, gives a significantly lower turnaround time on the whole.

This is what we do at Expand AI.

Here’s what our pipeline looks like.

Once we get data from clients, we label 5%-10% of the data manually. We then use this data to train the model to become domain-specific so that it can predict labels for the rest 90%-95% of the data accurately.

We have humans in the loop to review the labels to ensure quality. Finally, the labeled data is sent back to clients.

Reach out to us at client.success@expand-ai.com in order to test our data labeling software today!