Peer-reviewed evidence on predictive lead scoring is rare. Vendors publish claims, practitioners publish anecdotes, and almost nobody publishes data. So when a real study appears, with a real CRM dataset and real numbers, it deserves a careful read rather than a headline skim.
This is that meticulous read. The paper is González-Flores, Rubiano-Moreno, and Sosa-Gómez (2025), “The relevance of lead prioritization: a B2B lead scoring model based on machine learning,” published in Frontiers in Artificial Intelligence in March 2025. The authors built a machine learning lead scoring model for a B2B software SME using four years of Microsoft Dynamics CRM data, benchmarked fifteen classifiers, and reported near-perfect discrimination. It is one of the only peer-reviewed case studies of predictive lead scoring on real B2B CRM data, and it is worth your time. We have cited its results on this blog more than once, which is one more reason it deserves the closer read.
It also leaves three questions open. They are the three questions you should ask of any scoring model, including one you build or buy yourself. That is what makes the paper useful twice: once for what it shows, and once for what it prompts you to check.
What the study did
The setup is familiar to anyone who has worked in a Marketing Ops/RevOps seat. A B2B company selling product-design and manufacturing software used a manual, points-based scoring model (webinar attended: 5 points, demo request: 10, use of CTA: 35) and a sales team with low trust in the queue. Reps picked leads by intuition instead. Conversion stayed low.
The authors took 23,154 lead records with 67 fields from the company’s CRM, covering January 2020 to April 2024. After cleaning and feature selection, this became 16,600 records and 22 fields. The target variable was “qualified opportunity”: a binary label, heavily imbalanced, with 11.84% positives.
They then compared fifteen classification algorithms through PyCaret, from Naive Bayes to gradient boosting, using a 70/30 train/test split and 10-fold cross-validation. Gradient Boosting won: accuracy 0.9839, AUC 0.9891, recall 0.9586, F1 0.9338. The top features driving predictions were “Lead Source,” “Reason for Status,” and “Lead Classification,” followed by Product, Number of Responses, Account Type, and Interest Level.
What the paper gets right
Credit first, because the study does several things most vendor content never bothers with.
It benchmarks instead of asserting. Fifteen algorithms under one preprocessing pipeline, with a dummy classifier as the floor. The finding that gradient boosting methods (Gradient Boosting, LightGBM, XGBoost) cluster at the top while simpler models trail is consistent with what practitioners see on tabular CRM data, and it’s why boosted trees are the default for this problem class.
It takes class imbalance seriously. The authors tested SMOTE oversampling, found it did not improve AUC or F1 on this dataset, and kept the natural imbalance rather than applying the fashionable fix reflexively. That is the right order of operations: measure, then decide.
It checks its own learning curves. The paper inspects training-versus-validation curves, names the mild overfitting it finds, and notes that cross-validation scores improve with more data. Most scoring content never shows a learning curve.
It frames scoring as a prioritization problem, not an automation problem. The stated goal is ordering the queue so reps spend their limited attention on the leads most likely to progress. That framing matches how scoring actually earns adoption.
Question one: what does “Reason for Status” know, and when did it know it?
Now the journal-club part. The model’s second most important feature is “Reason for Status,” a CRM field that records why a lead’s status is what it is. In the paper’s own correlation matrix, Reason for Status has a correlation of -0.50 with the target variable, the strongest relationship to the target of any variable shown, by a wide margin.
Think about when that field gets filled in. A “reason for status” is typically written at or after the moment someone sets the status. If the status in question is qualification, then the field partially encodes the outcome the model is supposed to predict. The same concern applies to “Lead Classification,” another top feature, if classification is assigned during the qualification process rather than before it.
This failure mode is called target leakage: a training field that exists, or gets its value, only because the outcome already happened. A model trained on leaking fields looks brilliant in evaluation and collapses in production, because at scoring time, on a fresh lead, the field is empty or means something different. We wrote a practitioner explainer on exactly this pattern, target leakage and why we flag it, because it is the single most common way lead scoring models fool their builders.
To be fair to the authors: the paper does not publish per-field timestamps, so an outside reader cannot prove leakage from the text. The field may be populated early for some statuses. But an AUC of 0.9891 with a Kolmogorov-Smirnov statistic of 0.953 describes near-perfect class separation, and honest lead scoring models on messy CRM data do not usually live there. In our experience, AUC in the 0.75 to 0.85 range is a strong, deployable result (what AUC actually tells you). When a scoring model reports 0.99, the first hypothesis should not be “we solved lead scoring.” It should be “which field peeked at the answer?”
The practical takeaway for your own stack: for every field in your training set, ask when it gets populated relative to the outcome. If you cannot answer that from your CRM’s audit history, assume the worst until you can.
Question two: is “qualified” the outcome you actually care about?
The model predicts qualified opportunity, not won business. That is a defensible choice for a queue-ordering tool. But the paper’s correlation matrix contains a quietly important number: the correlation between “Qualified Opportunity” and “Opportunity Won” is 0.26.
Read that again. In this company’s own data, the label the model predicts has a weak relationship with the outcome that pays the bills. A model can be excellent at predicting qualification and still tell you little about revenue, because qualification is a human decision with its own biases, and those biases become the ceiling of what the model can learn. If reps qualify by source and title, the model will learn to predict source and title, with a scientific-looking probability attached.
This is why the training target matters more than the algo. Training on conversion outcomes, wins and losses alike, anchors the model to what the business is actually optimizing. It is a slower signal that requires more history, but it is the difference between predicting what your process does and predicting what your market does.
Question three: would the model survive a temporal split?
The study randomly splits its data 70/30 across records spanning January 2020 to April 2024. A random split over four years means the model trains on leads from 2024 and is tested on leads from 2020, and vice versa. It also means seasonal patterns, pricing changes, a pandemic, and any CRM process changes are smeared evenly across both sets.
The stricter test for a scoring model is temporal: train on the past, validate on the future. Models that look strong under random splits routinely degrade with temporal splits, and that gap is itself diagnostic. It tells you how fast your conversion patterns drift, and therefore how often the model will need retraining. The paper’s learning-curve analysis gestures at this by noting that more data helps, but a temporal holdout would have answered: will this still work next quarter?
What this means for your scoring stack
The study’s core conclusion holds and agrees with the broader literature it cites, including a 44-study review finding that scoring models consistently improve conversion rates and reduce qualification costs: machine learning beats manual point systems for ordering a B2B lead queue, and gradient boosting is the right default family. If you are still hand-assigning 35 points to a CTA click, the evidence is against you.
But the three open questions are the difference between a model that demos well and a model that survives contact with next quarter’s pipeline:
- Audit every feature for leakage. When does the field get its value, relative to the outcome? Status fields, reason fields, and classification fields are the usual suspects.
- Train on the outcome you want more of. Qualification labels inherit the biases of the people doing the qualifying. A recorded conversion does not.
- Validate forward in time. A random split will answer “can the model separate classes?” A temporal split answers “will it work on the next round?”
These are, not coincidentally, design decisions we made core to ax1om: the pipeline screens candidate fields for target leakage and identifies them before training, models train on your own conversion history, wins and losses alike, rather than on intermediate labels, and every score ships with its feature-level factors so a suspicious result can be interrogated instead of trusted blindly. Not because the authors of this paper got it wrong, but because four years of watching scoring models falter in production teaches you exactly where they break.
Papers like this one move the domain forward by putting real numbers on the table. The right response is neither to dismiss the 0.99 nor to repeat it in a sales deck. Read the correlation matrix, ask the three questions, and hold your own scoring model to the same standard.
Reference: González-Flores, L., Rubiano-Moreno, J., & Sosa-Gómez, G. (2025). The relevance of lead prioritization: a B2B lead scoring model based on machine learning. Frontiers in Artificial Intelligence, 8, 1554325. doi.org/10.3389/frai.2025.1554325