(also posted to r/datascience but I realized this community is almost 10x bigger)

Hey all,

I’m a data scientist and looking to reddit for some advice… I’ve been in this role for about two years and have been the only data scientist that entire time – this was also my first data scientist role. Since entering into the new ‘data’ team about a year ago, we’ve been continuously plagued with issues like:

• strange and distracting projects by our boss (who has no analytics experience, but a long career of software developer management) – and a lack of him be able to understand scope and true effort required for these random projects
• lack of interest in hearing from Sr DAs and me on what our ideal working environments would be (warehouse design, what we can put on our VMs, etc)
• lack of him working with leadership to build an actual understanding of business needs
• exerting random/arbitrary control over how things get done (I’ve never seen him do this and it end up being a benefit to a project)

I’ve stayed in the role mainly because it was my foot in the door to this industry (which I am very grateful for), and at the beginning (and to this day, really) the amount of possibilities here are huge and exciting… if they could ever be executed properly as a team. And, to add difficulty to it even more, my boss is an overall great guy – I just don’t think he has the mental horsepower for such a huge change this late into career.

My main predicament is that I’ve been tasked with building out a customer engagement ‘engine’ of sorts – so attempting to predict individually customers likely to leave, and also understand customer cohorts that are more engaged. I’m approaching it similar to a customer churn model, but with a few differences. This has been hyped for a year- the board of directors is aware of it, and so is everyone in leadership. To say the least: the hype around the team he was supposed to build is huge. That pendulum is beginning to swing back in.

The problem is that because of my manager’s disjointed priorities, we have had no progress in building out a warehouse or pipeline that helps a data scientist/me in any way. So, I’m spending time crafting ETL around extremely messy and unreliable system data which has cost me a few months just to implement that – and it isn’t done, of course. He has made very little progress in figuring out that his mental model of what a data scientist does is mostly not true and that the slowness of this project is a result of the past year’s lack of a decent strategy.

And just for a quick, very cringe, example: a few weeks ago he was sweet-talked by a Harvard MBA type into trialing yet-another-vendor’s autoML solution, thinking that the reason why my project has taken months to get off of the ground was simply because I was having trouble building decent models (I haven’t been able to train a model, here, for months because the data didn’t exist in any usable way!). And this is *not* because I’ve been quiet about the real challenges – he simply does not listen to other points of view unless it’s coming down to him from his boss.

But – to be fair – I made several mistakes when joining his team- my #1 one being that I doubted my intuition around the team’s strategy. If someone has 10+ years experience managing software teams, he’s this confident, and I’m this new to the role, then I need to stop challenging the status quo he’s putting together.

Recently I learned that – finally – the screws are coming down on him from his boss and he’s been told to do several of the things I suggested months ago (petty for me to mention, but it feels good and is validating). But rather than him reaching out to me for advice on changing strategy, or what we can do to accelerate the project, it’s more of the same. It’s also too late at this point for me to give suggestions that our small team could do before the end of the year.

Skip to here if you don’t wanna read:

Previously, I’ve said to myself that I will stay at this company until I can put into production at least one model. The question I’m coming to is, what do I do when the timeline for that keeps extending indefinitely and you’ve lost faith in management to be where you need them to be? What do I do in my job hunt when they see I’ve been in a DS role for ~2 years and never got to implement a model into production? If I was a hiring manager, I’d assume to some extent that this person wasn’t in a real data scientist role and would doubt my skills/abilities. Of course I could just lie in an interview- but that feels extremely gross.

My solution so far is to do a few ambitious personal projects that flex on modeling, python ability, and creativity. But we all know that (at least traditionally), your professional experience is the most important factor.

So, if anyone has words of encouragement, discouragement, suggestions, whatever- I’d love to hear it. I doubt my situation is truly unique – and I also know things could be much worse. I am thankful to have gotten my foot in the door, which can be quite hard.

Thanks for reading