Titanic Resurrected#
Recently, I have been playing around with a competition on Kaggle named Spaceship Titanic.
So far, I have tried 3 different models:
ensemble of
logistic regression
random forest
k nearest neighbors
xgboost
neural networks
But what’s exciting about it is that the third model (neural networks) turned out to be a pretty impressive solution! I got an accuracy of 0.80851, which is (as of today) within the top 200 on the leaderboard.
In fact, the model is pretty basic:
3 hidden layers
cross entropy loss
adam optimizer
dropout
l1/l2 ElasticNet regularization
Nevertheless, its stupidly good performance made me realize how good neural networks are at solving these kinds of classification problems. And I didn’t even need a GPU to train my model!
From an engineering perspective, I have two other things to talk about.
The first thing is that I have tried out Weights and Biases, which just like Tensorboard, helps you keep track of ML experiments. The visualization is pretty great! But the loading time is a bit too long. (perhaps too much javascript?)
The second is Pytorch Lightning. Indeed, I must admit that lightning did help me reduce my engineering workload, but the API design is still a bit unsatisfactory, at least in my opinion. Various methods are bunched together in a class, which makes it really messy. Anyway, I’m going to try out Pytorch Ignite and see which one is better.