Do you need a Data Engineer before you need a Data Scientist?

As demand for Data and Analytics roles continues to increase, we are seeing more diversity of roles and job titles. It does sometimes seem that the growing variety confuses candidates, as well as hiring managers, more than it helps. But, as a recruiter, we are not surprised to see the need for more diversity.

Placing many Data Scientists, as well as still Analysts – we hear what helps them succeed and stay with a company. It can feel like, too often, these highly skilled people are frustrated by their data environment. Having been hired for their analytical minds and specialist coding skills, quite a few data scientists later tell us they are spending most of their time working on data gaps or trying to improve data quality. You will know better than us, but is that true for your business?

Amongst the bewildering range of Data Journalists, Data Artists, Data Curators, Data Storytellers and Data Evangelists – perhaps there are a couple we’ve seen make a critical difference. In fact, the two roles I’m going to mention have often saved the productivity of Data Scientists, freeing them up to get on with what they’re good at.

Here are two job titles I’d encourage you to think about for your company or career

Data Engineers in the business

Recent research by CrowdFlower confirmed that today’s Data Scientists are still spending 80% of their time doing ‘data preparation’. That breaks down into spending 60% of their time cleaning and organising data, then 20% collecting the data sets they need.

Given that, in the same survey, Data Scientists stated that they view data preparation as the least enjoyable part of their work, you can see the problem. Would you stay in a job where you spend 80% of your time doing the work you enjoy least?

But, what is the answer? Many of our clients work in large organisations, with huge technology investments, legacy systems and centralised IT departments. These skilled people may be doing a good job delivering major projects and maintaining operational system security, but are rarely well positioned to help our frustrated data scientists.

This is where, it seems, Data Engineers can come to the rescue. Clients we have seen embrace this role, have someone within their Data Science or Analytics function who can tackle those data barriers. Focusing on what is now being called ‘Data Wrangling’, most Data Engineers have the coding skills and access rights to provide the data needed by their Data Science colleagues.

Whether they achieve this through Data Lakes, or ‘sandpits’ within larger Data Warehouses, they have expertise in extracting, transforming and loading data. Other candidates have proven the advantages of also mastering Data Quality Management and Metadata Management.

Having such a role within the business team, can help both IT and Data Scientist colleagues. But, it appears to work best when a strong partnership is formed with another ‘data role’.

Data Architects within IT

The title of Data Architect has been around much longer than others we have mentioned. Historically, it has grown up within centralised IT infrastructure or architecture teams. With the growing demands for integrated marketing, sales & CX systems, as well as customer data views, it is not surprising that a role has been needed to take a more strategic view of data technology.

Over a year ago, DataIQ ran an event called DataIQ Link, to which CIOs and CMOs were invited to come along together. Sitting in the audience, it was good to hear case studies from senior marketing and IT ‘double acts’. It brought home to me how much a strong partnership between Marketing or their Analytics teams and IT teams is needed to achieve desired results.

Talking with some of the Data Scientists and Data Engineers whom we’ve placed, it appears the same is true for these roles. The full promise of Data Engineers is realized when they work well with a Data Architect.

Talking with some CIOs and Data Architects, it appears the feeling is mutual. A few have found kindred spirits in Data Engineers within Analytics/Data Science teams. Having similar technical understanding and a common language (of both technology and data quality management terms), can help both communicate better what they need and cooperate to achieve success.

At MBN we encourage those creating or leading Data Science teams to build strong relationships with both the business terms they serve and their IT department. Perhaps Data Engineers and Data Architects should be recognised as some kind of ‘translator’ double act, spanning IT and Business? If you haven’t already, I’d encourage you to get these two-people talking.

I hope that advice ‘from the marketplace’ is useful, as you design your data science team, or consider which role is right for your career.

Given how many different job titles bombard our in-boxes, I thought it might also be helpful to share this quick visual summary of the 3 roles we’ve highlighted (plus the still needed data analysts).

Hopefully this table, published by Better Buys, can act as a useful aide memoir of the roles you may need within your Data Science team.

How are you designing your Data Science team?

As more businesses invest in Data Science teams, they need to decide on the roles required. I encourage you to think about all four of the roles listed above. Consider not only the exciting end-product you seek (persuasive beautifully presented analysis or accurate targeting models), but also what those analysts/scientists need. Could it be that your first priority should be hiring a Data Engineer & introducing them to your Data Architect?

If you’d value the opportunity to talk over the team you need, or check up on the roles that are working in other businesses, do get in touch. MBN is always happy to help. It’s only by working closely with our candidates and employers that we can help, together, to create a new family of roles that genuinely make a difference.

Michael Young – CEO – MBN Solutions – www.mbnsolutions.com 

MichaelY