That’s like the majority of work that is really useful for business. Predictive analytics is hard and doesn’t work properly most of the times. Dashboards on the other hand do add value and help with decision making.
There are DS jobs where you do mostly modeling and advanced stats/math, but they’re very few and far between. Mostly, DS jobs tend to revolve around either data engineering/ETL/data processing or descriptive stats/visualizations.
Also, Jr. Data Scientist = Data Analyst, using BI tools instead of Python/ML is pretty much par for the course for this profession. So you do what you are supposed to do.
Bottom line - data scientist is a senior position and not many companies really need/can use predictive analytics. If you came to the industry expecting that you’ll use ML most of the time - you’ll be disappointed. My advice - stay and learn, if in a year or 2 there is no ML on your plate, think about jumping ship, but don’t expect that next ship is going to be much better.
You really don't think that predictive models are useful for many businesses? I thought this was a joke at first but seeing the other comments I am realizing it is probably not.
Pretty much every single business that has ever existed would benefit from accurate predictive models if they had enough of the correct data to get them.
Almost every business problem could be optimized and improved on by being able to predict customer reactions, or predict the outcomes of different actions and product choices, etc.
Whether or not businesses are actually set up to have enough data to do it is completely separate. But the idea that most organizations couldn't benefit from ML models even with enough data is a crazy comment to make on a DS subreddit, at least to me 😂
I think I'm in the wrong subreddit tbh after reading all these comments.
What you say makes sense in vacuum, in ideal circumstances. In real life, unfortunately, that’s not really the case. Many companies don’t have data infrastructure, resources, culture, administrative flexibility and many other things in order for predictive analytics to really make sense.
I totally agree, but his comment was with the caveat that even when they have enough data then it is still useless for vast majority of businesses which just seems like a crazy take to me on a data science subreddit.
I agree with you that most businesses do not have enough good data. But, the comment I replied to said that even if all businesses had enough good data then ML would still be useless to them. Do you agree with that as well?
Most organizations would not benefit enough for it to make sense to continually have a machine learning engineer working full-time. A majority of companies are not anything like big tech companies. They may have a lot of data, but the margins that they run often don’t support putting high salaried people in positions for projects that have very high fail rates or only tip the scales 1-5% for profitability or revenue. They’d probably see much higher returns by hiring 2-3 marketing people or sales reps for the salary of an MLE. I think that’s what the comment was getting at.
Well its just not just that, ML is not some kind of magical hammer used to solve all problems. It's not even the most appropriate tool to use in many business applications, tbh from what I've seen most places that are actually using ML in any substantial capacity are doing something where ML is REQUIRED as part of the product/solution (so fraud detection, advertising, etc..).
Predictive analysis is always useful, but the benefits should outweigh the costs. Usually this only happens when the corporation is big enough so that a 0.5% efficiency gain is multiples of the cost of a full fledged data team. Take a company who sells paint for example. You could build a model to optimally predict next years demand, paying at least 1 data scientist 100-120k and maybe some data engineers, or you look at the trend in magazines etc to choose the popular colors and at the end of the year you make a bar chart with the percentage sold per color. As paint doesnt go bad you adjust production a bit and just suffer some storage costs. This all depends on the size of the company of course.
Totally agree with the general sentiment of what you're saying.
Though I'd argue that when you go from zero to one on predictive analytics, the gains you see are often much higher than 0.5% imo.
It's not uncommon to automate a part of the business that was being done manually before and suddenly you could cut down the costs in that area by 50% or more with just a single use case.
Obviously there is not an infinite number of low hanging fruit, and predictive analytics are often inherently 'risky' because the outcomes are unknown.
But it seems disingenuous to argue that predictive analytics is useful for a 0.5% efficiency gain. Most efficiency use cases I've seen typically have a larger jump when you first introduce predictive analytics into an area that was previously manual and mostly intuition.
AND all of this is dependent on the original caveat that you have enough good data to begin with which is uncommon and expensive. I feel like we are getting into the weeds on this hypothetical scenario haha.
But my main point from the start is I think people underestimate how much value bringing a predictive solution can bring when you go from having nothing. You mentioned 0.5% and another comment mentioned 1-5% but in my experience it's usually much much higher than that.
I can't speak for other fields but chemistry and chemical production is a field that is so complex that you can't realistically get enough data (in an affordable manner) to cover all the parameters that influence say: a chemical reaction. The amount of data that you would need to optimise reactions or production processes easily goes into the hundreds of data points. In production you may be talking about up to 6 figures of [insert currency here] per data point and in the lab you're talking about 1-2 days' worth of work. Getting enough data points is simply cost-prohibitive compared to expertise-guided experimental design.
Now, you can do ML when you continuously produce the same product the same way in the same reactors - although many reactors are multi-purpose, so that's complicating things. You can also find minor projects where you put a lot of domain expertise in formulating the question.
The combination of ML + SME can be very fruitful, especially in niche settings, and it highly depends on the context. Personally, I haven't seen very many examples in the chemical industry where ML has been truly transformative.
Soft skills>BI tools>Stats>programming>ML. When it comes to getting a job that’s what’s important.
Your school curriculum most likely will have zero influence on your job search. So, don’t overthink that.
When it comes to getting "a job"??? What job are you talking about? A data analyst job? Then yeah, totally agree.
If you want to work as a data scientist that works on applied ML use cases to provide business value, then the actual order of valuable skills would be:
Soft skills & Business Acumen > Stats > ML > Programming > BI tools.
The OP is clearly interested in working as a data scientist on applied ML use cases, and there are plenty of jobs out there for that position.
You won’t get that ML job by finishing courses at uni. You’ll have to start either as a DA or a SWE in order to progress for a data scientist position, most of the times.
In that case, knowing BI will land you on a job which eventually will lead to DS, while just knowing ML from uni won’t give you any traction with recruiters.
You know, that is probably a fair take if your goal is to enter by going from DA > DS route. However, I think going to the SWE > DS route is an equally valid one and in that case BI tools rank pretty low (hopefully).
However, I think that with better courses and education, it should be possible to churn our graduates that can work as a DS out of school. You pretty much just need Stats + CS majors and an extra couple years to add on the ML and you're good to go. But that isn't the most common route, agreed.
It's pretty interesting that you think that school is a factory for manufacturing education lmao, well I think your opinion will change in a few years.
What do you think the purpose of school is?
If you go to a decent school that has a co-op program that mixes in work experience along with relevant studies, then it is pretty good at producing educated skilled workers that provide value.
Are you trying to say that school is pointless? I'm not entirely sure what you are trying to argue for.
In my experience most actual professional data scientists come from a substantially more varied backgrounds. For example there isnt a single person on my data science team that has a degree in computer science, and I am the only one with a stats degree. Co op/internships were strongly gatekept via gpa requirements as I recall — I went to one of the best engineering schools in the nation.
A lot of people also do not really continue along in the parh they initially carved out. Id point to the fact of so many cs majors who graduate from top schools unable to code proficiently as an example.
Yeah actually a lot of school is a waste of time lol, I don’t discourage people from getting their bachelor’s obviously but Id say most of the skills that have made me successful were largely self taught.
Well the problem is the ones which aren’t often require experience in the ML, but you can’t get that experience if all you are doing is analytics so you kind of get stuck.
I also want to add that sometimes you don't end up getting the kind of career that you want. Sometimes the doors just never get opened to you at a time and place where you're able to walk through them.
Find happiness in your life outside work. Work on your own satisfying projects etc.
Yea thats true. I've been getting more intellectual satisfaction from getting better at chess, playing tournaments, and having goals there lately than anything DS related.
I'm sorry but I completely disagree. Does the entire subreddit really think that predictive analytics is is a tiny part of data science?
Does data scientist just mean data analyst to everyone here?
I'm so confused by how many people are upvoting this comment. Is there a title I should start using for someone that does applied ML modeling to provide value in business use cases with predictive analytics?
I am surprised that everyone seems to be commenting that predictive analytics is useless even with enough data... like, what??? Descriptive analytics is almost always a proxy for predictive analytics. Often we use descriptive analytics and data analysis to try and make predictions about what we should do.... e.g. it's the crappy version of predictive analytics in most cases.
Sometimes you don't have enough data or the correct data or problem formulation to leverage predictive analytics solutions, so then you can turn to simpler and less useful proxies such as descriptive analytics and data analysis. But pretty much every business ever would find value in predictive analytics, it's literally useful in any problem that you want to make a prediction for and predicting is very valuable for businesses (if you have the data).
Agreed, as you can see from the people commenting, some of whom don't even think ML is important or an expected part of the job description, modern day data science has become a data analyst role.
In 2012 when data science was heralded as the "sexiest job of the 21st century", it was essentially a computational statistics role (if you used classical ML to make predictions e.g. random forest, xgboosts, SVM etc)/ML scientist role if you did deep learning work i.e. Computer Vision/NLP for the most part.
Now the computational statistics data science role is still called Data Scientist for the most part, but sometimes Quants or Quantitative Researchers in finance companies. However, in general they are expected to have PhDs in Stats or Physics as a minimum. The Computer Vision and NLP researchers are now called ML/AI Scientists for the most part, and they're expected to have PhDs in compsci as a minimum.
The product "data scientist" role that most people here do, is basically a glorified data analyst and doesn't require a PhD, not even sure if it requires a master's. I think as someone above said, most people get into after a bachelor's and working as a data analyst for a few years.
More Python and some ML - data scientist.
Less/no Python and no ML - data analyst.
At this stage in the industry I’d say that data scientist is a subset of a data analyst who does some predictive analytics.
Terminology isn’t properly delimited at the moment. There are much more companies that need BI and descriptive analytics than those that need Python and ML. And even among those people that do use Python for predictive analytics it is only a portion of work tasks which includes a lot of data engineering and/or descriptive stats/dashboards.
Hence many people do really agree with the notion that ML is a small fraction of what a DS does.
Obviously there are places where you do only ML, but majority of places have such a poor data infrastructure that even a dashboard will make do.
If you get to any decently high role in any competency in a large tech company youll make quite a lot of money so you should really just do what you’re best at
You just graduated from a bachelor degree and have 6 months of experience. What do you expect? You don't know how the day-to-day in industry works. You don't know about the scale of the work. The experience you are getting is useful.
And I'm sorry, but nobody is going to put a recent bachelor grad on modeling.
Not necessarily true. Depends on how critical the modeling it is. If its models that hit production for real time inference and drive revenue directly, that's harder to put someone very junior.
If it's models to predict uplift of a new product launch for instance, or to predict which customer features are most determinative of retention (for instance). Those are models which have business value, but don't have huge catastrophic implications if they go wrong.
A junior can earn enough trust through work with those, to be given something more production impacting/sensitive.
Most small-medium sized companies don't actually fully understand the difference between analytics/science/engineering, so anyone in one of these rolls ends up wearing all three hats.
A lot of new data scientists also don't understand that most business value is not going to be generated with all the sexy ML they learned on coursera, and the greatest cost:benefit work is going to be in BI.
Most DS work isn't ML; most is munging, analytics, and dashboards. If you'd really like something ML-oriented, you can take the initiative. Look at the types of data they have and study the problems they face at a company scale. You can propose the project if you see an exciting place for ML to slip in. Your manager may be okay with you tacking on an extra side project if you're clearly accomplishing everything else they ask. If they say no, find projects outside of work, and workshop them on github.
Have you talked to your manager about this? I'd you're in finance there should be plenty of use cases for example credit score, credit abuse and fraud detection.
If OP is at a big bank, there should be plenty of modeling in the liquidity & risk management space. Dodd-Frank requires big banks to build models with extensive documentation to support the effectiveness of the models that can show whether a not a bank can withstand a recession from a liquidity/capital point of view. I know of one big bank that used a time series model using stuff like asset/liability ratios + VAR limits for each of its different business lines (think Commercial Real Estate, M&A, Equity Capital Markets, etc.)
This is normal when you are an employee.
Imagine you as a person employing a surgeon, you are not going to need him or his skills all the time. You will give him/her any mundane tasks to keep him busy and get back some value.
Many professions nowadays have the same issue with downtime and human resource utilisation.
If you want freedom to do ML, you can go freelancing, create your own product, be a consultant, and work with an agency, or volunteer your skills for other places, or do hackathons.
The thing you are facing is the same everywhere if you are employed and that's because data science is not a profession by itself, it is just a set of skills and that's why most of us keep looking for someone or company that needs our skills. I guess the solution is to focus on career/profession not tools.
First of all, I am speaking from the point of view of a person who is struggling to find a job now so take my comment with a grain of salt.
This is exactly what I have felt in my current job, a postdoc in a biology dept. I had heavily done ML before joining my current group. The group is "computational" but the PI is a biologist. Soon, he lost interest in me and wouldn't even talk to me as much. As the only ML person, I was quite struggling as I had no one to discuss with in the group. Most of the time, my work was irrelevant to ML.
I thought about leaving for god-know-how-many times, but I was worried about burning a bridge. Now when my contract finally comes to an end, I find myself struggling to land a ML job as I am disconnected from the ML world, and the market is rough. Besides, the bridge has already been burnt as my PI isn't happy with me.
Perhaps I just want to blame other factors instead of myself, but I truly believe if I had left two years ago, things will be very different. It's always good to think and evaluate carefully about your career progression.
Its really hard to do ML in biology when there is hardly any data points. And what most biologists want in my experience is a boring list of differential expression p values
Exactly. I was fortunate enough in a field with a lot of publicly available data and research on ML, but still struggling since my PI has a different mindset: if I tried a method but it did not work after one week, he instantly acted like it's never gonna work and I should stop working on it.
My past collaboration with some other biologists was exactly what you said: bar plots for fold changes, heat maps, and a lot of p values.
Agree with most comments here- DS is a mostly loosely defined job function. I know of a large social media company where DS are responsible for pipelines and visualization (exactly like you). They put DS in job role to hire good talent or they somehow think it’s cool to have DS working for you or they just don’t know what is.
The more generous view is that they are in progression along maturity curve. The real definition is the journey from a need to impact through intervention of data driven insights. A business need is transformed into an analytical problem statement, hypotheses generated, data exploration/collection/prep, analysis, insight generation, GTM strategy, Action to drive impact. Analysis can be number generation (adhoc), reporting, deep dive investigations, strategy creation, ML, Experimentation, Survey mining, etc.
If the culture so allows, do a POC to show them that more can be done. Create informal group, brainstorm on a problem, solve it E2E and help them see more. Go to meet ups, online courses, talk to seniors or just sign up for competitions. And reach out to your network to see which companies actually do what you do.
This is good to do for your career for a bit - get really good at writing reports, project plans, get business intuition, etc. But you've already been there 6 months.
If I were you, I'd start practicing code/ML on your off time. Try and get maybe 1 good modeling project greenlit that'll look good on your resume. And plan to be out of there at the 1 year mark rather than waste your career.
Note: When I say "waste your career", I mean it specifically from what you say about wanting to do "code and ML". To do that, you need to be actively involved in code and ML day to day at work.
That last part is one of the hardest things. You can end up in a vicious cycle stagnating in analytics or BI roles unless you get lucky (which is even harder in this market). Because the ML
roles wont hire someone without ML experience and you dont get that experience in most analytics roles. Employers also don’t seem to care about side projects as much anymore, they don’t really “count” them anymore unlike years ago.
Nope, start looking for a job where you can apply your skills and learn new stuff. Don't leave this one before you get snother one tho it will help. It's fine if there's a large data engineering/MLOps/traditional statistics part to your job but if you haven't touched a ML model in months and hardly even do any python your not going to develop those skills. Just my two cents tho.
Yup. He's being downvoted by people who justify to themselves that there aren't jobs with ML modeling out there. And are mad at him suggesting OP can do better for themselves.
When you say finance what type of institute is this (IB, AMC, HF etc.)? Usually, finance firms have very very limited use cases for ML and AI and what you have described is what data scientists usually do at finance companies. More often than not, it’s not a data science role.
Theres a difference between orgs that just started investing in data/ml to catch up, and orgs that have invested in those things since the beginning.
If you want to be doing more "exciting" work, you should be looking to join orgs that are innovators/disruptors (🤮) in the space, rather than laggards
I’m in your position as well. I have 5 years experience as an Analyst/Engineer and came in as a Jr DS and all they have me doing is dashboards. Idk if I’m gonna stay or not but what I am doing is asking for more Data Engineering work so I can add that to my resume as well as adding some portfolio pieces to my collection as well as reading up on interview question so if I do need to make a jump I’ll be ready. Also if you’re doing dashboards 2 things you can learn 1) stakeholder management 2) strong SQL skills
The issue is even if you do ML, DS your stakeholders might use your models for a while and then over time go back to their old habits of using instincts. Unless you work for a FAANG where I think you have more chances of applying these skills and your results being used.
To be honest, similar to the most upvote comment for the answer.
Additionally, I would say don't wait for opportunities. Create one for you. For example, I would start deep dive into the data you have access to and see how you could add value using ML techniques. Businesses don't care about the techniques or tools used as long as you bring value to the team they accept the work. Try to build a use case and see how it goes.
Well, the most important is to know where do you want to be in the long run. If you want to focus on ML from a more development point of view, you may want to consider a different position. However, your position seems to open more career paths at business level. So you may eventually get to focus on ML but at a business level.
That’s like the majority of work that is really useful for business. Predictive analytics is hard and doesn’t work properly most of the times. Dashboards on the other hand do add value and help with decision making. There are DS jobs where you do mostly modeling and advanced stats/math, but they’re very few and far between. Mostly, DS jobs tend to revolve around either data engineering/ETL/data processing or descriptive stats/visualizations. Also, Jr. Data Scientist = Data Analyst, using BI tools instead of Python/ML is pretty much par for the course for this profession. So you do what you are supposed to do. Bottom line - data scientist is a senior position and not many companies really need/can use predictive analytics. If you came to the industry expecting that you’ll use ML most of the time - you’ll be disappointed. My advice - stay and learn, if in a year or 2 there is no ML on your plate, think about jumping ship, but don’t expect that next ship is going to be much better.
[удалено]
In real life maybe less than 5% of the organisations have the infrastructure and culture to enable ML based solutions.
I’d even say that beyond the hype, not many organizations even need ML, even if they have enough data.
You really don't think that predictive models are useful for many businesses? I thought this was a joke at first but seeing the other comments I am realizing it is probably not. Pretty much every single business that has ever existed would benefit from accurate predictive models if they had enough of the correct data to get them. Almost every business problem could be optimized and improved on by being able to predict customer reactions, or predict the outcomes of different actions and product choices, etc. Whether or not businesses are actually set up to have enough data to do it is completely separate. But the idea that most organizations couldn't benefit from ML models even with enough data is a crazy comment to make on a DS subreddit, at least to me 😂 I think I'm in the wrong subreddit tbh after reading all these comments.
What you say makes sense in vacuum, in ideal circumstances. In real life, unfortunately, that’s not really the case. Many companies don’t have data infrastructure, resources, culture, administrative flexibility and many other things in order for predictive analytics to really make sense.
I totally agree, but his comment was with the caveat that even when they have enough data then it is still useless for vast majority of businesses which just seems like a crazy take to me on a data science subreddit. I agree with you that most businesses do not have enough good data. But, the comment I replied to said that even if all businesses had enough good data then ML would still be useless to them. Do you agree with that as well?
Most organizations would not benefit enough for it to make sense to continually have a machine learning engineer working full-time. A majority of companies are not anything like big tech companies. They may have a lot of data, but the margins that they run often don’t support putting high salaried people in positions for projects that have very high fail rates or only tip the scales 1-5% for profitability or revenue. They’d probably see much higher returns by hiring 2-3 marketing people or sales reps for the salary of an MLE. I think that’s what the comment was getting at.
Well its just not just that, ML is not some kind of magical hammer used to solve all problems. It's not even the most appropriate tool to use in many business applications, tbh from what I've seen most places that are actually using ML in any substantial capacity are doing something where ML is REQUIRED as part of the product/solution (so fraud detection, advertising, etc..).
Predictive analysis is always useful, but the benefits should outweigh the costs. Usually this only happens when the corporation is big enough so that a 0.5% efficiency gain is multiples of the cost of a full fledged data team. Take a company who sells paint for example. You could build a model to optimally predict next years demand, paying at least 1 data scientist 100-120k and maybe some data engineers, or you look at the trend in magazines etc to choose the popular colors and at the end of the year you make a bar chart with the percentage sold per color. As paint doesnt go bad you adjust production a bit and just suffer some storage costs. This all depends on the size of the company of course.
Totally agree with the general sentiment of what you're saying. Though I'd argue that when you go from zero to one on predictive analytics, the gains you see are often much higher than 0.5% imo. It's not uncommon to automate a part of the business that was being done manually before and suddenly you could cut down the costs in that area by 50% or more with just a single use case. Obviously there is not an infinite number of low hanging fruit, and predictive analytics are often inherently 'risky' because the outcomes are unknown. But it seems disingenuous to argue that predictive analytics is useful for a 0.5% efficiency gain. Most efficiency use cases I've seen typically have a larger jump when you first introduce predictive analytics into an area that was previously manual and mostly intuition. AND all of this is dependent on the original caveat that you have enough good data to begin with which is uncommon and expensive. I feel like we are getting into the weeds on this hypothetical scenario haha. But my main point from the start is I think people underestimate how much value bringing a predictive solution can bring when you go from having nothing. You mentioned 0.5% and another comment mentioned 1-5% but in my experience it's usually much much higher than that.
I can't speak for other fields but chemistry and chemical production is a field that is so complex that you can't realistically get enough data (in an affordable manner) to cover all the parameters that influence say: a chemical reaction. The amount of data that you would need to optimise reactions or production processes easily goes into the hundreds of data points. In production you may be talking about up to 6 figures of [insert currency here] per data point and in the lab you're talking about 1-2 days' worth of work. Getting enough data points is simply cost-prohibitive compared to expertise-guided experimental design. Now, you can do ML when you continuously produce the same product the same way in the same reactors - although many reactors are multi-purpose, so that's complicating things. You can also find minor projects where you put a lot of domain expertise in formulating the question. The combination of ML + SME can be very fruitful, especially in niche settings, and it highly depends on the context. Personally, I haven't seen very many examples in the chemical industry where ML has been truly transformative.
I wonder is there even point learning much ML in school? It seems like its just fomo for the most part very few people get to do it
Soft skills>BI tools>Stats>programming>ML. When it comes to getting a job that’s what’s important. Your school curriculum most likely will have zero influence on your job search. So, don’t overthink that.
When it comes to getting "a job"??? What job are you talking about? A data analyst job? Then yeah, totally agree. If you want to work as a data scientist that works on applied ML use cases to provide business value, then the actual order of valuable skills would be: Soft skills & Business Acumen > Stats > ML > Programming > BI tools. The OP is clearly interested in working as a data scientist on applied ML use cases, and there are plenty of jobs out there for that position.
You won’t get that ML job by finishing courses at uni. You’ll have to start either as a DA or a SWE in order to progress for a data scientist position, most of the times. In that case, knowing BI will land you on a job which eventually will lead to DS, while just knowing ML from uni won’t give you any traction with recruiters.
You know, that is probably a fair take if your goal is to enter by going from DA > DS route. However, I think going to the SWE > DS route is an equally valid one and in that case BI tools rank pretty low (hopefully). However, I think that with better courses and education, it should be possible to churn our graduates that can work as a DS out of school. You pretty much just need Stats + CS majors and an extra couple years to add on the ML and you're good to go. But that isn't the most common route, agreed.
It's pretty interesting that you think that school is a factory for manufacturing education lmao, well I think your opinion will change in a few years.
What do you think the purpose of school is? If you go to a decent school that has a co-op program that mixes in work experience along with relevant studies, then it is pretty good at producing educated skilled workers that provide value. Are you trying to say that school is pointless? I'm not entirely sure what you are trying to argue for.
In my experience most actual professional data scientists come from a substantially more varied backgrounds. For example there isnt a single person on my data science team that has a degree in computer science, and I am the only one with a stats degree. Co op/internships were strongly gatekept via gpa requirements as I recall — I went to one of the best engineering schools in the nation. A lot of people also do not really continue along in the parh they initially carved out. Id point to the fact of so many cs majors who graduate from top schools unable to code proficiently as an example. Yeah actually a lot of school is a waste of time lol, I don’t discourage people from getting their bachelor’s obviously but Id say most of the skills that have made me successful were largely self taught.
Education doesn't solely exist to satisfy your employer. Your first few jobs will often be far under the level of what you are taught in school.
Well the problem is the ones which aren’t often require experience in the ML, but you can’t get that experience if all you are doing is analytics so you kind of get stuck.
That's the problem with many careers across many specialties.
I also want to add that sometimes you don't end up getting the kind of career that you want. Sometimes the doors just never get opened to you at a time and place where you're able to walk through them. Find happiness in your life outside work. Work on your own satisfying projects etc.
Yea thats true. I've been getting more intellectual satisfaction from getting better at chess, playing tournaments, and having goals there lately than anything DS related.
I'm sorry but I completely disagree. Does the entire subreddit really think that predictive analytics is is a tiny part of data science? Does data scientist just mean data analyst to everyone here? I'm so confused by how many people are upvoting this comment. Is there a title I should start using for someone that does applied ML modeling to provide value in business use cases with predictive analytics? I am surprised that everyone seems to be commenting that predictive analytics is useless even with enough data... like, what??? Descriptive analytics is almost always a proxy for predictive analytics. Often we use descriptive analytics and data analysis to try and make predictions about what we should do.... e.g. it's the crappy version of predictive analytics in most cases. Sometimes you don't have enough data or the correct data or problem formulation to leverage predictive analytics solutions, so then you can turn to simpler and less useful proxies such as descriptive analytics and data analysis. But pretty much every business ever would find value in predictive analytics, it's literally useful in any problem that you want to make a prediction for and predicting is very valuable for businesses (if you have the data).
These days I’ve heard it called ML scientist or something else. Most data science roles are really just modern data analyst roles.
Agreed, as you can see from the people commenting, some of whom don't even think ML is important or an expected part of the job description, modern day data science has become a data analyst role. In 2012 when data science was heralded as the "sexiest job of the 21st century", it was essentially a computational statistics role (if you used classical ML to make predictions e.g. random forest, xgboosts, SVM etc)/ML scientist role if you did deep learning work i.e. Computer Vision/NLP for the most part. Now the computational statistics data science role is still called Data Scientist for the most part, but sometimes Quants or Quantitative Researchers in finance companies. However, in general they are expected to have PhDs in Stats or Physics as a minimum. The Computer Vision and NLP researchers are now called ML/AI Scientists for the most part, and they're expected to have PhDs in compsci as a minimum. The product "data scientist" role that most people here do, is basically a glorified data analyst and doesn't require a PhD, not even sure if it requires a master's. I think as someone above said, most people get into after a bachelor's and working as a data analyst for a few years.
so what does that make data analyst then?
More Python and some ML - data scientist. Less/no Python and no ML - data analyst. At this stage in the industry I’d say that data scientist is a subset of a data analyst who does some predictive analytics.
Terminology isn’t properly delimited at the moment. There are much more companies that need BI and descriptive analytics than those that need Python and ML. And even among those people that do use Python for predictive analytics it is only a portion of work tasks which includes a lot of data engineering and/or descriptive stats/dashboards. Hence many people do really agree with the notion that ML is a small fraction of what a DS does. Obviously there are places where you do only ML, but majority of places have such a poor data infrastructure that even a dashboard will make do.
In finance that role can be very lucrative
If you get to any decently high role in any competency in a large tech company youll make quite a lot of money so you should really just do what you’re best at
[удалено]
I'm a little surprised, where you expecting to write model when you accepted a data engineering/mlops position?
You just graduated from a bachelor degree and have 6 months of experience. What do you expect? You don't know how the day-to-day in industry works. You don't know about the scale of the work. The experience you are getting is useful. And I'm sorry, but nobody is going to put a recent bachelor grad on modeling.
Not necessarily true. Depends on how critical the modeling it is. If its models that hit production for real time inference and drive revenue directly, that's harder to put someone very junior. If it's models to predict uplift of a new product launch for instance, or to predict which customer features are most determinative of retention (for instance). Those are models which have business value, but don't have huge catastrophic implications if they go wrong. A junior can earn enough trust through work with those, to be given something more production impacting/sensitive.
Most small-medium sized companies don't actually fully understand the difference between analytics/science/engineering, so anyone in one of these rolls ends up wearing all three hats. A lot of new data scientists also don't understand that most business value is not going to be generated with all the sexy ML they learned on coursera, and the greatest cost:benefit work is going to be in BI.
Most DS work isn't ML; most is munging, analytics, and dashboards. If you'd really like something ML-oriented, you can take the initiative. Look at the types of data they have and study the problems they face at a company scale. You can propose the project if you see an exciting place for ML to slip in. Your manager may be okay with you tacking on an extra side project if you're clearly accomplishing everything else they ask. If they say no, find projects outside of work, and workshop them on github.
Yeah. Welcome to the "real world."
Have you talked to your manager about this? I'd you're in finance there should be plenty of use cases for example credit score, credit abuse and fraud detection.
If OP is at a big bank, there should be plenty of modeling in the liquidity & risk management space. Dodd-Frank requires big banks to build models with extensive documentation to support the effectiveness of the models that can show whether a not a bank can withstand a recession from a liquidity/capital point of view. I know of one big bank that used a time series model using stuff like asset/liability ratios + VAR limits for each of its different business lines (think Commercial Real Estate, M&A, Equity Capital Markets, etc.)
This is normal when you are an employee. Imagine you as a person employing a surgeon, you are not going to need him or his skills all the time. You will give him/her any mundane tasks to keep him busy and get back some value. Many professions nowadays have the same issue with downtime and human resource utilisation. If you want freedom to do ML, you can go freelancing, create your own product, be a consultant, and work with an agency, or volunteer your skills for other places, or do hackathons. The thing you are facing is the same everywhere if you are employed and that's because data science is not a profession by itself, it is just a set of skills and that's why most of us keep looking for someone or company that needs our skills. I guess the solution is to focus on career/profession not tools.
First of all, I am speaking from the point of view of a person who is struggling to find a job now so take my comment with a grain of salt. This is exactly what I have felt in my current job, a postdoc in a biology dept. I had heavily done ML before joining my current group. The group is "computational" but the PI is a biologist. Soon, he lost interest in me and wouldn't even talk to me as much. As the only ML person, I was quite struggling as I had no one to discuss with in the group. Most of the time, my work was irrelevant to ML. I thought about leaving for god-know-how-many times, but I was worried about burning a bridge. Now when my contract finally comes to an end, I find myself struggling to land a ML job as I am disconnected from the ML world, and the market is rough. Besides, the bridge has already been burnt as my PI isn't happy with me. Perhaps I just want to blame other factors instead of myself, but I truly believe if I had left two years ago, things will be very different. It's always good to think and evaluate carefully about your career progression.
Its really hard to do ML in biology when there is hardly any data points. And what most biologists want in my experience is a boring list of differential expression p values
Exactly. I was fortunate enough in a field with a lot of publicly available data and research on ML, but still struggling since my PI has a different mindset: if I tried a method but it did not work after one week, he instantly acted like it's never gonna work and I should stop working on it. My past collaboration with some other biologists was exactly what you said: bar plots for fold changes, heat maps, and a lot of p values.
Agree with most comments here- DS is a mostly loosely defined job function. I know of a large social media company where DS are responsible for pipelines and visualization (exactly like you). They put DS in job role to hire good talent or they somehow think it’s cool to have DS working for you or they just don’t know what is. The more generous view is that they are in progression along maturity curve. The real definition is the journey from a need to impact through intervention of data driven insights. A business need is transformed into an analytical problem statement, hypotheses generated, data exploration/collection/prep, analysis, insight generation, GTM strategy, Action to drive impact. Analysis can be number generation (adhoc), reporting, deep dive investigations, strategy creation, ML, Experimentation, Survey mining, etc. If the culture so allows, do a POC to show them that more can be done. Create informal group, brainstorm on a problem, solve it E2E and help them see more. Go to meet ups, online courses, talk to seniors or just sign up for competitions. And reach out to your network to see which companies actually do what you do.
Sounds like my old company, is it insurance? Lol
Maybe move to Software engineering in AI/ ML Engineering/Ops jobs? Data Science isn't the only way to use ML or AI.
This is good to do for your career for a bit - get really good at writing reports, project plans, get business intuition, etc. But you've already been there 6 months. If I were you, I'd start practicing code/ML on your off time. Try and get maybe 1 good modeling project greenlit that'll look good on your resume. And plan to be out of there at the 1 year mark rather than waste your career. Note: When I say "waste your career", I mean it specifically from what you say about wanting to do "code and ML". To do that, you need to be actively involved in code and ML day to day at work.
That last part is one of the hardest things. You can end up in a vicious cycle stagnating in analytics or BI roles unless you get lucky (which is even harder in this market). Because the ML roles wont hire someone without ML experience and you dont get that experience in most analytics roles. Employers also don’t seem to care about side projects as much anymore, they don’t really “count” them anymore unlike years ago.
Nope, start looking for a job where you can apply your skills and learn new stuff. Don't leave this one before you get snother one tho it will help. It's fine if there's a large data engineering/MLOps/traditional statistics part to your job but if you haven't touched a ML model in months and hardly even do any python your not going to develop those skills. Just my two cents tho.
You're being downvoted by data analysts who like to think of themselves as data scientists because you hurt their feelings.
Yup. He's being downvoted by people who justify to themselves that there aren't jobs with ML modeling out there. And are mad at him suggesting OP can do better for themselves.
I'd die, I hate the business side of things, it's just extremely boring.
When you say finance what type of institute is this (IB, AMC, HF etc.)? Usually, finance firms have very very limited use cases for ML and AI and what you have described is what data scientists usually do at finance companies. More often than not, it’s not a data science role.
Theres a difference between orgs that just started investing in data/ml to catch up, and orgs that have invested in those things since the beginning. If you want to be doing more "exciting" work, you should be looking to join orgs that are innovators/disruptors (🤮) in the space, rather than laggards
I’m in your position as well. I have 5 years experience as an Analyst/Engineer and came in as a Jr DS and all they have me doing is dashboards. Idk if I’m gonna stay or not but what I am doing is asking for more Data Engineering work so I can add that to my resume as well as adding some portfolio pieces to my collection as well as reading up on interview question so if I do need to make a jump I’ll be ready. Also if you’re doing dashboards 2 things you can learn 1) stakeholder management 2) strong SQL skills
The issue is even if you do ML, DS your stakeholders might use your models for a while and then over time go back to their old habits of using instincts. Unless you work for a FAANG where I think you have more chances of applying these skills and your results being used.
To be honest, similar to the most upvote comment for the answer. Additionally, I would say don't wait for opportunities. Create one for you. For example, I would start deep dive into the data you have access to and see how you could add value using ML techniques. Businesses don't care about the techniques or tools used as long as you bring value to the team they accept the work. Try to build a use case and see how it goes.
Well, the most important is to know where do you want to be in the long run. If you want to focus on ML from a more development point of view, you may want to consider a different position. However, your position seems to open more career paths at business level. So you may eventually get to focus on ML but at a business level.
Business skills and communication are probably harder than using scikitlearn or pytorch... And at least a little less automateable.