[D] Design methodology for ML engineering.
I am currently preparing for ML Engineer interviews and wanted to learn a design methodology to crack the design rounds. I have looked for standard resources but couldn’t find any. So I thought of coming up with one on my own from my own experiences and material that I have read online. So below is an overview of the approach I have in my mind. Please let me know what you think of it.
- Understand the use case.
- Figure out what can be solved deterministically
- Figure out what needs to be solved using ‘Data + Machine learning’
- Pose the problem(1.b) as a math problem.
- Come up with an objective that you want to optimize.
- Select the right data set.
- What data is needed. How is the labeling done/derived?
- How do you deal with bias, skewed classes,
- Feature Engineering.
- Think about what(information) you need to solve the problem optimally.
- What aspects from the data could you use to replicate findings in 4.a?
- Transform data into features and a form more apt for model’s learning
- Model selection:
- What model is best suited to solve the problem(2.a) at hand.
- Are there any off the shelf(direct or transfer learning) that would help.
- Training:
- How will you train
- How will you handle: skewed classes and other problems associated with the dataset.
- Validation: what metrics, experiments do you conduct to validate the model’s learning.
- Productionizing:
- How will the solution be deployed?
- Performance monitoring, feedback loop/retrain,
- Application scaling and model maintenance.
submitted by /u/kireeti_
[link] [comments]