In 2015, I was offered corporate funding for a PhD at the University of Illinois Urbana-Champaign (UIUC). Further, I had a professor interested in working with me and an interesting area of research.
Instead, I decided to pursue a career in industry with my recently issued B.S. in Computer Science.
I joined Capital One.
Qualifications
During my five years with the company I’ve built a multitude of machine learning models and systems. Much of my work eventually made it into production. I have worked on: Fraud detection, insider threat detection, synthetic data generation, a deep learning & hyperparameter tuning platform, AutoML, sensitive data detection and more…
I’ve also amassed hundreds of submitted & issued patents. Have thousands of co-workers attend my teams trainings on deep learning annually. At the moment, I am lead on an applied research team and we have and continue to deliver multiple high value projects (some listed above). I even regularly commit code to my github (often in private repos) outside of work.
Job Descriptions
Even with the qualifications above — today, I am still unqualified for many roles in the machine learning and data science space, as I don’t have an “advanced degree”. I also don’t have multiple years of managerial experience. On paper I’m essentially a full-stack engineer.
Like everyone, I have a unique skill set, often it doesn’t fit in a box. In the “tech industry” there are clearly defined roles:
- front-end engineer
- back-end engineer
- full-stack engineer
- data engineer
- data scientist
- machine learning engineer
- engineering manager
- product manager
In my case, I’d classify myself as some where between a “full-stack engineer”, “product manager” & “machine learning engineer”. The truth is, our team (myself included) regularly takes a product from inception (discussing with customers) to a useful deliverable, typically in a few months. Perhaps “consultant” or “entrepreneur” is a better term for what I do. We’ve been often told we act like a startup in the company.
Unfortunately, there often isn’t this role at a company; the exception being an early stage startup.
This leaves me wondering: is there a better way? Shouldn’t we fill multiple roles as needed?
Overqualified
I am often approached for these “full-stack software engineering positions”. Frankly, it feels a lot like being approached for a job in food delivery. They are usually decent pay, but “full-stack software engineers” are a commodity. At this point, I’ve built at least ten medium or larger Ruby on Rails applications. It’s not fun for me. It’s easy-ish. I’m sure I can build what you want quickly with documentation, tests, and stability.
However, I can probably provide 10x more value in other ways…
Underqualified
At the same time, I don’t have an PhD. I am never approached for data science roles and only occasionally for machine learning engineering positions.
When I interview for machine learning engineering roles I’m often given specific statistics questions, often in a pop-quiz style format. E.g. “What is p-value”. The closest questions to machine learning are looking for linear regression or Naive Bayes classifiers as the desired answer(s).
I don’t know if this is appropriate, machine learning engineers are often optimizing neural network models or implementing a distributed algorithm. Perhaps this is because most companies I speak with don’t have machine learning engineers yet, so they have data scientists conducting interviews. I don’t know, but it doesn’t seem correct.
Unfortunately, statistics is not my forte or interest. I took statistics a decade ago and although I use it weekly, I’ll often just look up what I need to when I need it. I can answer questions about p-values or “what’s correlation vs covariance” or basic questions, but much beyond that – I need reference material. My search-foo is fairly strong, so why memorize.
My personal opinion, is data science / machine learning roles should focus on a combination of generalized knowledge and “how to approach the problem”. How one approaches the problem is likely more important than specific knowledge and probably provides greater insights to team fit & problem solving. At least, that’s what I look for when hiring co-workers.
In the end, the status quo makes it difficult to pass these interviews. Importantly, if an outright rejection does not occur due to a lack of PhD, the process is still bias. The interviewer is targeting someone who has a large corpus of knowledge, while virtually ignoring the finesse required for problem solving. Interviewers often seem they are on more of an ego-trip, rather than a test of whether or not the interviewee can solve the problems faced day-to-day.
Intrapreneurship
All that being said… I enjoy my job at Capital One. My team carved out a niche and in essence crafted our roles to suite us. I personally manage / develop entire product life-cycle(s), providing me vital experience and keeping my job interesting. While it’s often fairly stressful, it (in my opinion) is preparing me for my own entrepreneurship. It’s also the only instance of intrapreneurship I’ve witnessed.
Bringing me to my final thoughts. Perhaps it’d be better to hire people based on their skill sets, provided they are highly motivated [to achieve organizational objectives]. From what I’ve seen, most people want to be useful and some people strive to be great at anything they do. If you can collect people with a high amount of motivation and some general knowledge they’ll build useful things. Perhaps some skill sets will be per-requisite, but generally we shouldn’t dig into the weeds.