No. Not exactly. There are two things going on. First, there are places which have a large enough supply of problems which are clearly machine learning problems the that it makes sense that industry is producing so-called machine learning engineers. The idea is that if you can reduce the job scope of some area, you can train people faster and pay them less money. This is good for companies clearly and possibly good for people that can’t possibly learn all that is required to be a data scientist.

In reality, this was inevitable and I’ve seen this coming along time. There are never going to be enough data scientists for industry so they will break the job apart as best they can and train people to do these more constrained job functions. That solves some of the problem. It allows you to apply data science for problems fitting several well defined patterns and get a process in place for delivering to production. This isn’t a bad thing.

However, that does not mean they will replace data scientists. Machine learning engineers are only going to solve the problems that are classified correctly as machine learning problems. It’s like when you go to a surgeon. They are probably going to recommend surgery. A data scientist is more like the general practitioner. They have a far broader scope and are in a better place to decide on what specialty is needed. They might just recommend exercise and a better diet rather than rush into a risky and costly surgery.

The truth is the field of data science has become too focused on machine learning. I think that’s just because it’s easier to teach. There are some great libraries and some strong repeatable patterns such as supervised ML or image classification. The downside of this is that many junior data scientists are learning very little else. The big thing they lack is an understanding of statistics and more specifically understanding of statistical information. They rely too much on trial and error where a good data scientist who understands the problem better at a theoretical level will just start at the right solution and be more productive. They are less likely to bring in new ideas from other fields because they don’t see the mathematical essence of the problem as well.

As for the future of these careers… there will always be an intense need for data scientists. This is the domain of big picture thinking and modeling the world in an analytical fashion. It requires broad knowledge and creativity. It is one of those things that humans will always be better at than computers. The skill set needed is high, and so high demand and low supply should result in this staying a good career for the rest of our lives at least.

I don’t feel the same about machine learning engineer. The concept of machine learning engineer is an attempt by industry to patternize and specialize a job function which is a way of making it less creative and more automatable. In addition, most of the technology being developed is aimed at this corner of data science increasing the productivity of the practitioners.

While today there is still plenty of demand for this I fear this will not always be the case. Higher productivity is good for the employee when it is not outpacing demand for the service. Industry will do everything it can to identify overlapping work and strive for cost savings. Companies will spring up with speciality products that can serve many companies having the same problem. This reduces the need for companies to have in-house machine learning skills. To use the medical metaphor again; today they are surgeons but industry hopes to make them into nurses. I suspect supply will be able to keep up with demand in this area much more easily and I can imagine a time not too far off where salaries for these kind of jobs match those of general software developer but lags behind those of data scientists.

Leave a Reply

Your email address will not be published. Required fields are marked *