I'm trying to train a classifier with neural network, but I've got too small datasets. Each class has about ~1k examples. What is the best approach?
Solution 1
There is a paper called "Overfitting Mechanism and Avoidance in Deep Neural Networks" by Shaeke Salman and Xiuwen Liu for the cases when you have a small dataset. I thought they do something like ensembling, but the approcah is different. They rely on the examples, which are classified with high probobility, like different models are confident for these examples. The other part called sensetive examples, because different models give different results. Take a look at the paper