Torchtitan is an excellent project to learn how to implement distributed training techniques for training massive language models on hundreds of GPUs. I’ve been using it for a few months, and now that the paper is out, I thought it’d be a good idea to share a few posts about how to use it, what works and what doesn’t, what I learned while implementing these ideas in my own project.
BLAST (Basic Local Alignment Search Tool) is a widely used tool for comparing nucleotide or protein sequences against databases. If you’re working on macOS and need to use any of its tools, this guide will help you install BLAST, download necessary databases, and run BLAST searches efficiently.
Installing BLAST manually gives you control over the installation process and allows you to install the latest version directly from NCBI.
In this blog I will share my thoughts on machine learning, although some other topics might pop up. I will try to be as concise as possible and avoid all the unnecessary jargon *cough* academia *cough*. I hope you not only enjoy the content but also learn something new.
Disclaimers:
Well, we have neural networks, they are awesome, they work but there’s a problem. THEY ARE HUGE. We scaled from a hundred of millions of parameters to hundred of BILLIONS. This problem makes using neural networks in real life quite hard as you normally don’t have this huge computational capabilities to run them anywhere.
Neural networks have proven to be a very valuable tool in scenarios where the transformation from inputs to outputs is unknown. Suppose you are asked to write an algorithm to classify an image if it’s a cat or a dog, how would you do that ? Well first you might ask yourself, “what makes an image a cat?”. Answering this question is incredibly hard because a vast amount of cases to cover in order to have your algorithm generalizable. This is where neural networks shine; Given an input $ x_{i} $ with its respective label $ y_{i}$ you can use a neural network model with a set of parameters $\theta$ denoted by $ M(\theta) $ to approximate $y_{i} = f(x_{i})$. Normally with enough data you can get a very good estimate of $f$. However, this comes at a huge cost, training and running these large networks is expensive in terms of time and memory because of the huge amount of parameters that you need to learn to get the best approximation, this makes these models hard to use in real life scenarios. Also, the recent trend of models getting bigger and bigger in order to get better performance is making this problem even harder.
So finally Pytorch is getting a decent (?) production serving capabilities. TorchServe was introduced a couple of days ago along with other interesting things
So TorchServce was announced as a “industrial-grade path to deploying PyTorch models for inference at scale”. In this tutorial we will try to load a finetuned BERT model.
What we want to do is learn the mapping between the angular rotation (complete circle) of a circular object given its diameter to the distance covered on a linear path
