Introduction

How do computer vision techniques work in an industry setting? How does an organization use data engineering to scale up its operations?

These are questions every aspiring data scientist must be aware of. Dat Tran, Head of Data Science at idealo internet GmbH, is the perfect person to shed light on these questions.

Dat has worked on a variety of data engineering projects before he came to idealo, and now leads a team of data scientists who work on really cool computer vision problems. This is one of my favorite episodes since we launched DataHack Radio — the depth and breadth of topics covered, plus Dat’s incredible knowledge, make this a must-listen.

In this episode of the DataHack Radio podcast, Kunal and Dat cover multiple topics, including:

I have penned down a few highlights from the podcast below. But I strongly recommend listening to the entire conversation! The energy Dat brings to this episode is incredible.

You can subscribe to the DataHack Radio podcast on any of the below platforms:

Dat’s journey into data science isn’t your run-of-the-mill story. He hadn’t even heard of ‘machine learning’ during his undergrad days, where his focus was on investment banking. But Dat quickly realized it wasn’t the field for him. So what next?

Back to the drawing board — a Master’s degree! During this time, a couple of his friends were starting out in machine learning and it wasn’t long before Dat was drawn into this wonderfully complex field.

He landed a job in the advanced analytics department at Accenture. This was back when ‘Big Data’ was starting to become the ultimate buzzword in the industry — a great time to enter this field. Dat moved to Pivotal Inc. a year later (joining as a data scientist), recognizing that this was a brilliant opportunity to get more hands-on experience in machine learning.

At Pivotal, Dat worked on a variety of projects spanning different industries, including automotive and airlines. He worked there for over two years and credits a lot of his current knowledge and experience to his time at Pivotal. He gave talks at multiple PyData conferences as well during this time — a truly impressive achievement.

Dat is now working as the Head of Data Science at idealo internet GmbH, a successful Berlin-based startup and one of the largest portals in the German e-commerce market.

idealo is a price comparison site (for products as well as hotels) so you can imagine the numerous data science functions the team performs — price prediction, indexing, developing and using a recommendation engine, among other things. Dat’s team, however, focuses on applying computer vision.

A fair question to ask — what role does computer vision have in a price comparison site? Well, idealo has a ton of images of products and hotels:

Dat explained this section using a really intuitive example. idealo has approximately 2 million accommodations listed with 130 images per accommodation (on average). Now, there are all kinds of hotels — small-sized, medium ones, and the big players (the luxurious 5-star ones).

The pictures of these hotel rooms vary depending on who took them. The non-luxury hotels typically have images taken by owners themselves while the 5-star hotels send images taken by professionals. There is quite a big gap in the image quality between these two categories.

Dat and his team use an array of computer vision concepts to analyze and make use of these images:

Really interesting stuff! It’s a pleasure to see computer vision making inroads in the industry, isn’t it?

I came across Dat’s talk at PyData on YouTube — it doesn’t take long to realize he is a data engineering expert. His talk is on ‘How you really get your data science models into production the cool way!’ and you can check it out below:

At idealo, there are a variety of tools being used for data engineering, such as AWS for training and Kubernetes for putting models into production.

I personally feel data engineering is a very overlooked aspect (by aspiring data scientists) of the overall data science project lifecycle. You will most certainly face questions on model deployment and other aspects of software engineering in your data scientist interview. This section of the podcast will provide you with a bird’s eye view of an industry-ready process.

Data science and data engineering are inextricably linked — you cannot separate them for all intents and purposes. Dat explained this using the example of a neural network (a convolutional neural network (CNN), to be precise). There are quite a few CNN frameworks to choose from, like RESNET, MobileNET, VGG, etc.

The challenge with these CNN models is they have tons of hyperparameters, hence making them quite large. This brings up the age-old debate of balancing accuracy and speed. You can get away with it in research but when you’re working with production environments? That is a significant obstacle.

Dat mentioned quite a few common challenges from the data engineering specific side as well, including:

As we alluded to earlier, Dat has done most of his data science learning on the job. There is nothing like practical hands-on experience to indelibly ingrain concepts.

Outside of that, there are so many options to learn from these days (everything is a quick Google search away!):

A major challenge with these platforms is that we don’t get a structured path or answer to a specific problem. That, again, is why experience is king in data science.

Software engineering is a key facet of data science most aspiring professionals are unaware of. And you simply can’t get away from it in an industry role. So here’s Dat’s advice for you:

And this really, REALLY important point:

Knowing mathematics, statistics, machine learning algorithms and even tools like R and Python is good, but these don’t differentiate you from the competition. Everyone else is learning the same thing. So what else is there? It comes down to that one thing again — software engineering.

Dat uses a straightforward set of pointers and rounds to judge a candidate’s ability:

Which machine learning functions will see a major improvement and focus in the coming years?

One of my favorite DataHack Radio episodes so far! Dat brings a ton of enthusiasm and knowledge to the podcast that really shines through in the way he explains his role, the challenges his team faces from both a data science as well as a data engineering perspective, his advice to aspiring data scientists, among other things.

A pleasure listening to him elaborate on relevant industry problems and how to overcome them. What was your favorite part of the episode? Let us know in the comments section below.

Related posts:

When first beginning a career as an entrepreneur, it can be intimidating figuring out the path that you want to take, as well as determining the best way to begin. As a result of the unique approach…

Fui descoberta. Fui redescoberta. A pessoa que vive em mim há anos, está em transformação. O momento é tão intenso, profundo, que quase passou despercebido. Me reencontrei. Ou me encontrei, não sei…

Sleep is the single most important thing you can do to create a foundation for lasting helath and longevity. Restorative rest is the key to a happy, healthy and productive life; improving cognitive…