Machine learning / Research / Case study
Asteroid Risk Prediction
A recall-first classifier for potentially hazardous asteroids, built on NASA near-Earth-object data.
01 / Idea
Accuracy is the wrong question.
This project classifies asteroid close approaches as potentially hazardous or not, using NASA near-Earth-object data. It demonstrates classical machine learning independently of LLM systems.
Only about 9% of the records are hazardous, so a model can look accurate while missing most of them. The central question became: how many positives can be caught without turning every approach into an alarm?
02 / Stack
Make the method visible.
The data is the public NASA Near-Earth Objects dataset on Hugging Face (Hari5115/nasa-neows): 64,576 non-hazardous and 6,368 potentially hazardous records. The work runs in a Jupyter notebook with pandas, scikit-learn, XGBoost and LightGBM, on Python 3.14 managed by uv.
Imputation, scaling and encoding live inside the Pipeline, so they are fitted only on each training fold. The test split stays out of tuning and is used once, for the final evaluation.
Data
NASA NeoWs (Hugging Face) · pandas
70,944 close approaches, 9% labeled potentially hazardous; approach dates decomposed into numeric features.
Modeling
scikit-learn · XGBoost · LightGBM
Six model families compared inside the same leakage-free preprocessing pipeline.
Validation
RandomizedSearchCV · Stratified 3-fold CV
120 configurations ranked by a weighted score in which recall carries 55%.
03 / Architecture
Data, preparation, training, evaluation.
Identifiers and the orbiting body are dropped. The textual approach date becomes year, month and day of year instead of thousands of one-hot categories.
All six model families share the same preprocessing. A custom refit callback applies the project's weighted score, so each family's hyperparameters and the final model are chosen by the same priority.
01 / Responsibility
Scientific data
Absolute magnitude, estimated diameter range, Sentry flag, velocity, miss distance in km and lunar distances, and the approach date.
02 / Responsibility
Preparation
A ColumnTransformer inside the Pipeline: median imputation and scaling for numbers, most-frequent imputation and one-hot encoding for categories.
03 / Responsibility
Training
Logistic Regression, Random Forest, Extra Trees, HistGradientBoosting, XGBoost and LightGBM, each tuned over 20 random configurations.
04 / Responsibility
Evaluation
A 30% stratified test set held out of tuning, a confusion matrix and a configurable risk policy on top of predict_proba.
04 / Flow
The output is a score, not a verdict.
The winning pipeline returns a score through predict_proba, and a separate policy decides what to do with it: the model ranks objects by risk, the operational policy defines which ranges demand action.
Scores from class-balanced models are not calibrated probabilities. Calibration and threshold validation would come before any real use — this is an educational project, not a planetary-defense system.
- 01
Prepare
Drop identifiers, decompose the approach date and split 70/30 with stratification.
- 02
Train
Tune each family with RandomizedSearchCV over stratified folds, class weighting included in the search space.
- 03
Compare
Rank by a weighted score: recall 55%, average precision 25%, precision 8%, F1 7%, ROC AUC 4%, accuracy 1%.
- 04
Interpret
Map the score to low risk (under 30%), specialist review (30–75%) or a high-priority alert (75% and above).
05 / Challenges
There is no free improvement.
The first, unbalanced Logistic Regression looked accurate but missed 1,561 hazardous cases. Class weighting lifted its recall to about 94%, at the cost of precision near 31%. Tree ensembles reacted differently: Extra Trees stayed at 33% recall, and GradientBoosting was replaced by HistGradientBoosting so it could take class weights.
After tuning, Random Forest won with a weighted score of 0.7929 — only 0.001 ahead of Extra Trees. On the held-out test set it caught 1,896 of 1,910 hazardous objects (recall ≈ 99.3%, 14 false negatives), with 3,854 false positives and precision near 33%.
06 / Status
Classical ML, made inspectable.
The notebook, the model ranking and the confusion matrix are public in the repository, including the pre-tuning results that show how each decision moved the metrics.
Recall was bought with explicit false positives. The limits are documented: uncalibrated scores, a single dataset, and a test that is an experiment rather than validation for real-world use.
Evidence / From the repository
