Machine Learning: Baselines, Data and Generalization
Machine learning fits patterns from examples to make predictions or support decisions.
- Author
- absequ
- Published
- Updated
- Category
- AI and data
- machine learning
- fundamentals
- practical guide
Machine learning fits patterns from examples to make predictions or support decisions. Its value depends on performance on relevant new cases, not just the examples used during training. Before fitting a model, define a target, a baseline and the consequence of each kind of error.
On this page
Key concepts
In supervised learning, features describe an example and a label is the outcome to predict. Training adjusts the model; validation informs choices; a held-out test set supports a final assessment. Overfitting occurs when a model fits training examples but performs poorly on new ones.
Practical example
Imagine predicting whether a synthetic equipment reading needs inspection. Temperature and vibration at observation time can be features; the later inspection outcome can be a label. Do not include the inspection result in the inputs. If many readings come from one machine, keep related observations together when dividing data.
For 100 illustrative cases with 10 faults, a model that always predicts no fault is 90 percent accurate but detects none. Suppose another model flags 12 cases: 8 are faults and 4 are false alarms. Precision is 8/12 and recall is 8/10. Choose an operating threshold based on inspection cost and missed-fault impact, not accuracy alone.
Common mistakes
Data leakage makes evaluation look better than real use. Split before fitting preprocessing, and avoid repeatedly tuning against the test set. A random split can also mislead for time-dependent observations: train on earlier periods and evaluate on later ones when that matches deployment.
Practice
Start with a simple baseline and document where it fails. Compare metrics across meaningful operating conditions and watch for changing inputs after release. A model that needs too much manual correction may be less useful than an explicit rule.