NLP research
SetFit reproduction & paper-code audit
Reproducing few-shot classification and investigating a paper-to-code discrepancy.
Role: Independent implementation and evaluation
Reading the project context and repository evidence.
01
challenge
A reported machine-learning result is only useful when the experimental setup can be reproduced. Few-shot classification adds sensitivity to sampling and training configuration, so one successful run is insufficient evidence.
02
approach
Reimplement the SetFit workflow without the official SetFit package. Use deterministic sampling, contrastive sentence-encoder fine-tuning and downstream classification, then inspect the released source to compare implementation choices with the paper.
03
solution
The work combined a runnable training pipeline with repeated evaluation and controlled experiments. A discrepancy between the documented learning rate and the effective released implementation was investigated at source level and tested experimentally.
04
impact
The supplied CV reports 88.61 ± 1.61% accuracy across 10 independent CR MPNet runs, compared with the paper’s 88.5 ± 1.9%. The portfolio reproduces that CV claim; it does not present a new evaluation.