How to label testing dataset?

I have two cases for label training samples by using active learning:
Case 1: I would apply active learning to query a small chunk of samples gradually to label them and my model is being trained during this process. After a certain number of iterations, I have a training dataset with specific performance of the model.
Case 2: I re-train the model from scratch with the training dataset in case 1.

Question 1(regarding training): do you think the performance of the model will be the same in both cases? why, please?
Question 2(regarding testing): do you think there is a difference to apply active learning to label the testing dataset after case 1 or apply it after case 2?

1 Like

@adamc @Deep_Madi @Kenan_Morani do you have any experience with this approach and any advice?

1 Like

no sorry! Here’s a nice review of active learning in a deep-learning context which might be useful:

2 Likes

@H.Al-Qadasi , I’ve had a recommendation to look at Tom Mitchell’s NELL (Never-ending language learner). It’s a bit dated compared to modern approaches, but it gives some context in how you would build a more complex system to achieve active learning:

https://www.cmu.edu/homepage/computing/2010/fall/nell-computer-that-learns.shtml

1 Like
Category icons made by Freepik from www.flaticon.com