Well, don't paint with a broad brush. I love solving problems too, and I love learning new things. I too played with computers as a kid and my middle school friends and I built websites for fun. I wasn't exposed to real programming (beyond neat Javascript tricks) until CS class in high school.

However, science has always been my #1 thing, and programming is my probably my #3. Interesting, but not the only thing I want to do. Working 40 hours a week at it is enough for me. I still love learning new things and getting better at the craft, but when I go home, I have other hobbies.

I'm hacking my treadmill with Arduino for fun. I'm brazing metal. I'm riding my bike more. I'm reading a fiction book that just came out. I just read a book about world development. All of that is a lot more interesting to me than going home and sitting at a different desk doing more programming, you know?

And while I'm not offended by your comments and this is entirely non-personal -- really, I'm just curious as well and enjoy talking to people with different perspectives -- the attitude that if I'm not always programming, I'm "less than" than other programmers seems oppressive to me. I'm still smart, I'm still curious, I'm still good at my job. But there's still the perception that if I am not programming more, I am falling behind.

So the solution for people like me is to pretend to be someone like you, who loves programming above all other things, and it's mildly exhausting. It's great that you love programming so much and I kind of envy you. But not everyone needs to be exactly like you to be a worthwhile contributor. (Again, not personal, I just enjoy putting these thoughts into words.)

>the attitude that if I'm not always programming, I'm "less than" than other programmers seems oppressive to me.

Who says that? It's so backwards from everything I'm accustomed to in the bay area. (In the bay area I've done 10 years of data science and 5 years as a software engineer before that.) Maybe it's because I didn't go to universities out here and it's university culture? People at MIT can get very passionate about their projects.

I've had companies (many of them) advertise to me saying I should join them because they have a healthy 9 to 5 work culture. But what company doesn't? Maybe a young startup, but you sign up for that at that point.

Everyone I know who regularly works more than 40 hours a week (who isn't a CEO) is almost always because their boss asks for X quantity load of work done, which might be 50 hours of work, so they do the 50 hours instead of communicating they can't do that much within 40 hours. Sometimes it's because they're a junior and are afraid of not fitting in and are afraid of getting fired, but sometimes it's a senior who has their tech down, but not their communication skills down. It happens, and it's sad to see, because it almost always leads to burn out on their end. It's not a good thing.

I love what I do, but I do 40 hours a week. Want more? It better be a rare event (crunch times are a real thing in some industries) or you need to be paying me more, and I'm not cheap.

>So the solution for people like me is to pretend to be someone like you, who loves programming above all other things, and it's mildly exhausting.

I love programming, but it's just a tool. It's like saying you love hammering. It doesn't make a lot of sense when you take a step back. What I love is the projects I do. You mentioned hacking your tread mill, so it sounds like you do more tech hobby stuff after hours than most software engineers.

Frankly, I rarely do projects outside of work, despite loving what I do. Why? Because companies give me inspiration to do specific tasks that I love. Without working I have a hard time finding things to do. If I could find infinite things to do, I'd be a CEO, providing things for everyone else to do. The only people who truly don't turn off, end up being CEOs, not software engineers. (Unless they don't know how to capitalize, or some other edge case, of course.) So it's unrealistic to believe you have to be doing non-work tech to work in tech. That's just absurd.

Also, I can't say from country to country, but within the US software engineer culture, while it varies from company to company and team to team, is generally somewhat similar across the country. On the NYC side, working more than 40 hours a week is common (eg, quant work), where on the bay area side only working 40 hours a week and turning off (unless you're at a startup) is common.

Is it possible this has to do with too small of a sample size? It's easy to stereotype. We naturally do it unless we learn otherwise, sometimes through learning statistics, sometimes through other routes.

> You're on ycombinator, the center of Bay Area tech culture

Citation needed. I think YC users think they are the center of tech (or the universe?).

What would be that actual proof that it was? Do we know how many users HN has? What % of VC actually spend time here? What % of meaningful founders actually spend time here? Ya the CEO of github will drop a post on a major thread concerning his company, but so what?

Honestly the more time I spend here the more I see two notable demographics: cynics trying to feel superior (like myself), and people trying to justify their own salary and lifestyles via a fat FANNG paycheck.

P.S. and college students / recent grads who know everything.

It could be trending away from that, but historically https://www.ycombinator.com/ is the center of bay area startup culture, and then those startups grew into bay area tech culture as a whole.

Ycombinator comes from Stanford which centers around creating CEOs, creating most of the companies in the bay area. Most non-yc bay area companies still have Stanford at its roots. I don't have a direct data point, but almost every company I've interviewed for in the last 10 years and have worked with, their CEO or CTO or some other c-suite "club" is strongly tied to Stanford and from that directly or indirectly tied to YC.

What do you need to learn to make the switch? I know SQL pretty well. I don't now a lot of stats, but I doubt its beyond me to learn.

It's a multifaceted question and has a multifaceted answer, unfortunately.

There is how to get into a data science job, and how to make that switch.

There is how to get into doing data science as a hobby or a skill/experience, and how to make that switch.

There is how did you make the switch to data science. That is, what is your story?

And there is the deeper dive, most people do not consider enough: "Will I like doing data science work?"

My story:

When I was a teen, I wrote a stock market bot that was pretty profitable. That may be a quant researcher job and skillset, but it is a kind of data science too, as data science is a research position, where creating models is part of the job spec, identical to a quant researcher.

So I got into it, out of hobby, not out of taking a class or reading a book, but by being infatuated with the plots I see in the stock market, seeing patterns everywhere. When I'm doing a 9 to 5 I have to put a timer on when I'm looking at plots, or I can easily spend days just taking in information. I just like it, a lot.

In 2010 I applied for a job as a software engineer, passed all the interviews, was going to do another SE gig, and then I was talking to a manager during an interview and I said something like, "With enough time and effort, nothing is impossible. That's what I love about programming." So, he threw a curve ball at me. He said something like, "We have a team of roughly 15 people classifying web sites, labeling if the site is porn, education, business, and so on. There are a little over 60 categories." This team worked on the web filter schools and dmvs all over the country paid a service for, to limit what websites people can go to.

I jumped on it, "I can automate that!" So he started a new team and put me on it. What did I do? I looked up how SEOs work and how Google's search engine worked. Certainly they were classifying websites in one way or another, so it was just a matter of research and figuring it out.

...in the end the project succeeded. What strongly surprised me at the time was how it ended up categorizing websites to a higher degree of accuracy than the people we were paying.

That's the non-business side of data science in a nutshell. It's figuring out how to do something often suspected or assumed to be impossible, and making it possible. It's not just researching how to do something, but using cutting edge research from published papers and often doing the next that no one has figured out yet.

Programming is a tool that automates thoughts. If you can figure it out you can put it into code. If it's a complex enough of a problem you can use ML to automate the writing parts of that algorithm you could write by hand. ML is often thought of as a way to generate a process, that can't be written by hand, and to some extent that is true, but only in the sense that it would take too long to write by hand. Eg, where writing by hand would need an if statement for each of the millions of hypothetical scenarios.

ML is also useful for giving insights. You have a hypothesis -- an idea of how to do a task -- and you run ML over it, and either it solves the problem, or it doesn't, and what it comes up with that is incorrect will always teach you something, so data science is often an iterative loop, where you put some ML in just to see how the data varies from your hypothesis giving better ideas in how to write a better model.

Sometimes someone will say data science is an analyst + software engineer, but that's not quite right. An analyst looks at data, plots it, organizes it, and then reports on their findings about what is to management. A data scientist looks at findings and applies predictive analytics to it, finding solutions to potential problems. Eg, "We'd like you predict future customer churn, and propose solutions that will eliminate this potential future customer churn, before it happens." This is done through identifying or classifying what the causes of customer churn are, then finding solutions to those causes (hypothesis), then writing some software that automatically enacts that solution. Then you can look at if that solution is working, and validate your hypothesis or invalidate and learn more of what is really going on. Hopefully that makes sense.

In summary, if there is a task that other software engineers can't figure out how to do but you can, regardless if you're using ML or not, it may be data science. (Keep in mind, software engineers can use ML, so while data scientists often use ML, ML is not data science.) The path from software engineering to data science is usually just that, taking what is deemed impossible by those around you and making it possible.

How to get into DS as a career: As a software engineer, your best bet is to transfer internally. It's rare for a company to higher a fresh data scientist who previously was a software engineer. Data scientists can help all companies, from startups to large companies, so you don't have to be at the "right" company to switch, you just have to find a problem that could benefit the company, advertise your hypothetical solution to management, and start doing it. Data scientists are the ones finding projects to benefit the company, and doing it. They're usually not being given projects from management like software engineers are, though ofc they can be given projects too. Keep in mind: I'm a lead data scientist, and my first data science roll I ended up leading a data science team, so my views are biased towards my own experience.