I've recently been going through the lectures of oxford's 2017 deep nlp course (https://github.com/oxford-cs-deepnlp-2017). The course was well presented and I've really deepened my understanding of modern NLP methods.

Naturally I am going through the practicals as well. I've linked to the repo with my current progress but I feel a bit stuck atm.

The main task revolves around a multi-class classification of ~2k transcripts of TED talks. However, the dataset is heavily skewed with one class covering ~50% and some classes only around 3-5% of the data.

Practical 2 wants you to try a basic averaging over word-vectors approach and then pumping that through a single-hidden-layer NN. I've been trying to tweak a lot with preprocessing and tokenization but I can't come beyond ~66% accuracy on the test set.

In Practical 3 you are then supposed to try the same task with a RNN approach. I thought this might get better but I am basically stuck at around the same test set accuracy of ~66%.

Maybe not much more is possible, especially given the fact that there is very little data for some of the classes. Basically I am wondering if anyone else has gone through the course (or even attended the real deal at oxford) so we can get a discussion going.

Thanks in advance! //Michael