It's a multifaceted question and has a multifaceted answer, unfortunately.
There is how to get into a data science job, and how to make that switch.
There is how to get into doing data science as a hobby or a skill/experience, and how to make that switch.
There is how did you make the switch to data science. That is, what is your story?
And there is the deeper dive, most people do not consider enough: "Will I like doing data science work?"
My story:
When I was a teen, I wrote a stock market bot that was pretty profitable. That may be a quant researcher job and skillset, but it is a kind of data science too, as data science is a research position, where creating models is part of the job spec, identical to a quant researcher.
So I got into it, out of hobby, not out of taking a class or reading a book, but by being infatuated with the plots I see in the stock market, seeing patterns everywhere. When I'm doing a 9 to 5 I have to put a timer on when I'm looking at plots, or I can easily spend days just taking in information. I just like it, a lot.
In 2010 I applied for a job as a software engineer, passed all the interviews, was going to do another SE gig, and then I was talking to a manager during an interview and I said something like, "With enough time and effort, nothing is impossible. That's what I love about programming." So, he threw a curve ball at me. He said something like, "We have a team of roughly 15 people classifying web sites, labeling if the site is porn, education, business, and so on. There are a little over 60 categories." This team worked on the web filter schools and dmvs all over the country paid a service for, to limit what websites people can go to.
I jumped on it, "I can automate that!" So he started a new team and put me on it. What did I do? I looked up how SEOs work and how Google's search engine worked. Certainly they were classifying websites in one way or another, so it was just a matter of research and figuring it out.
...in the end the project succeeded. What strongly surprised me at the time was how it ended up categorizing websites to a higher degree of accuracy than the people we were paying.
That's the non-business side of data science in a nutshell. It's figuring out how to do something often suspected or assumed to be impossible, and making it possible. It's not just researching how to do something, but using cutting edge research from published papers and often doing the next that no one has figured out yet.
Programming is a tool that automates thoughts. If you can figure it out you can put it into code. If it's a complex enough of a problem you can use ML to automate the writing parts of that algorithm you could write by hand. ML is often thought of as a way to generate a process, that can't be written by hand, and to some extent that is true, but only in the sense that it would take too long to write by hand. Eg, where writing by hand would need an if statement for each of the millions of hypothetical scenarios.
ML is also useful for giving insights. You have a hypothesis -- an idea of how to do a task -- and you run ML over it, and either it solves the problem, or it doesn't, and what it comes up with that is incorrect will always teach you something, so data science is often an iterative loop, where you put some ML in just to see how the data varies from your hypothesis giving better ideas in how to write a better model.
Sometimes someone will say data science is an analyst + software engineer, but that's not quite right. An analyst looks at data, plots it, organizes it, and then reports on their findings about what is to management. A data scientist looks at findings and applies predictive analytics to it, finding solutions to potential problems. Eg, "We'd like you predict future customer churn, and propose solutions that will eliminate this potential future customer churn, before it happens." This is done through identifying or classifying what the causes of customer churn are, then finding solutions to those causes (hypothesis), then writing some software that automatically enacts that solution. Then you can look at if that solution is working, and validate your hypothesis or invalidate and learn more of what is really going on. Hopefully that makes sense.
In summary, if there is a task that other software engineers can't figure out how to do but you can, regardless if you're using ML or not, it may be data science. (Keep in mind, software engineers can use ML, so while data scientists often use ML, ML is not data science.) The path from software engineering to data science is usually just that, taking what is deemed impossible by those around you and making it possible.
How to get into DS as a career: As a software engineer, your best bet is to transfer internally. It's rare for a company to higher a fresh data scientist who previously was a software engineer. Data scientists can help all companies, from startups to large companies, so you don't have to be at the "right" company to switch, you just have to find a problem that could benefit the company, advertise your hypothetical solution to management, and start doing it. Data scientists are the ones finding projects to benefit the company, and doing it. They're usually not being given projects from management like software engineers are, though ofc they can be given projects too. Keep in mind: I'm a lead data scientist, and my first data science roll I ended up leading a data science team, so my views are biased towards my own experience.
There is how to get into a data science job, and how to make that switch.
There is how to get into doing data science as a hobby or a skill/experience, and how to make that switch.
There is how did you make the switch to data science. That is, what is your story?
And there is the deeper dive, most people do not consider enough: "Will I like doing data science work?"
My story:
When I was a teen, I wrote a stock market bot that was pretty profitable. That may be a quant researcher job and skillset, but it is a kind of data science too, as data science is a research position, where creating models is part of the job spec, identical to a quant researcher.
So I got into it, out of hobby, not out of taking a class or reading a book, but by being infatuated with the plots I see in the stock market, seeing patterns everywhere. When I'm doing a 9 to 5 I have to put a timer on when I'm looking at plots, or I can easily spend days just taking in information. I just like it, a lot.
In 2010 I applied for a job as a software engineer, passed all the interviews, was going to do another SE gig, and then I was talking to a manager during an interview and I said something like, "With enough time and effort, nothing is impossible. That's what I love about programming." So, he threw a curve ball at me. He said something like, "We have a team of roughly 15 people classifying web sites, labeling if the site is porn, education, business, and so on. There are a little over 60 categories." This team worked on the web filter schools and dmvs all over the country paid a service for, to limit what websites people can go to.
I jumped on it, "I can automate that!" So he started a new team and put me on it. What did I do? I looked up how SEOs work and how Google's search engine worked. Certainly they were classifying websites in one way or another, so it was just a matter of research and figuring it out.
...in the end the project succeeded. What strongly surprised me at the time was how it ended up categorizing websites to a higher degree of accuracy than the people we were paying.
That's the non-business side of data science in a nutshell. It's figuring out how to do something often suspected or assumed to be impossible, and making it possible. It's not just researching how to do something, but using cutting edge research from published papers and often doing the next that no one has figured out yet.
Programming is a tool that automates thoughts. If you can figure it out you can put it into code. If it's a complex enough of a problem you can use ML to automate the writing parts of that algorithm you could write by hand. ML is often thought of as a way to generate a process, that can't be written by hand, and to some extent that is true, but only in the sense that it would take too long to write by hand. Eg, where writing by hand would need an if statement for each of the millions of hypothetical scenarios.
ML is also useful for giving insights. You have a hypothesis -- an idea of how to do a task -- and you run ML over it, and either it solves the problem, or it doesn't, and what it comes up with that is incorrect will always teach you something, so data science is often an iterative loop, where you put some ML in just to see how the data varies from your hypothesis giving better ideas in how to write a better model.
Sometimes someone will say data science is an analyst + software engineer, but that's not quite right. An analyst looks at data, plots it, organizes it, and then reports on their findings about what is to management. A data scientist looks at findings and applies predictive analytics to it, finding solutions to potential problems. Eg, "We'd like you predict future customer churn, and propose solutions that will eliminate this potential future customer churn, before it happens." This is done through identifying or classifying what the causes of customer churn are, then finding solutions to those causes (hypothesis), then writing some software that automatically enacts that solution. Then you can look at if that solution is working, and validate your hypothesis or invalidate and learn more of what is really going on. Hopefully that makes sense.
In summary, if there is a task that other software engineers can't figure out how to do but you can, regardless if you're using ML or not, it may be data science. (Keep in mind, software engineers can use ML, so while data scientists often use ML, ML is not data science.) The path from software engineering to data science is usually just that, taking what is deemed impossible by those around you and making it possible.
How to get into DS as a career: As a software engineer, your best bet is to transfer internally. It's rare for a company to higher a fresh data scientist who previously was a software engineer. Data scientists can help all companies, from startups to large companies, so you don't have to be at the "right" company to switch, you just have to find a problem that could benefit the company, advertise your hypothetical solution to management, and start doing it. Data scientists are the ones finding projects to benefit the company, and doing it. They're usually not being given projects from management like software engineers are, though ofc they can be given projects too. Keep in mind: I'm a lead data scientist, and my first data science roll I ended up leading a data science team, so my views are biased towards my own experience.