Anyone done with the second problem (Fine tuning on Food100). Im getting 72.2% with 50,000 iterations. The highest I got on test was 75.6% after 9000 iterations and then it just goes downhill from there. So maybe this means that we are probably overfitting to the training set after about 10000 iterations.
The saturation in performance depends on learning rate and step size. I assume you are talking about part 2 of the assignment. If so, we have to set the same learning rate. What matters is the step-size parameter. Check the ratio of your step-size parameter with the max-iter parameter. If your step-size is big, reaching a maxima will be difficult. I agree about the overfitting point. I see a very slow increase in accuracy from 6000 to 10,000 iterations and my max accuracy was 72.6 % at 8000 iterations.
I am not observing any significant decline in the test accuracy once it reaches at the highest. The maximum accuracy I see is 74.08% at 30000 iteration and I get 74.02% at the end of 50000 iterations
Anyone done with the second problem (Fine tuning on Food100).
ReplyDeleteIm getting 72.2% with 50,000 iterations. The highest I got on test was 75.6% after 9000 iterations and then it just goes downhill from there. So maybe this means that we are probably overfitting to the training set after about 10000 iterations.
Has anyone tried dropout yet?
This comment has been removed by the author.
DeleteThe saturation in performance depends on learning rate and step size. I assume you are talking about part 2 of the assignment. If so, we have to set the same learning rate. What matters is the step-size parameter. Check the ratio of your step-size parameter with the max-iter parameter. If your step-size is big, reaching a maxima will be difficult.
DeleteI agree about the overfitting point. I see a very slow increase in accuracy from 6000 to 10,000 iterations and my max accuracy was 72.6 % at 8000 iterations.
I am not observing any significant decline in the test accuracy once it reaches at the highest. The maximum accuracy I see is 74.08% at 30000 iteration and I get 74.02% at the end of 50000 iterations
DeleteHas anyone tried using Python for creation of new layers?
ReplyDeleteIs it possible to use this for the PReLU?
Has anyone used Xavier initialization successfully? If so, what was your batch_size? I keep getting out of memory errors using GPU
ReplyDeleteThis comment has been removed by the author.
DeleteHow much RAM does your GPU has? The run with Xavier initialization is taking 2860 MB for my experiments. I am using batch size of 50.
DeleteI got Xavier initialization working with batch_size 100 i think.
ReplyDeleteWhat GPU are you using?
I have tried PReLU with slope variable for both x>0 and x<0.
ReplyDeleteNo improvement for the fine-tuning task :(