Training a LSTM network and sampling the resulting model in ml5.js

RNNs are very useful if your input is, for example, a corpus of text or a musical composition and you are trying to predict meaningful sequences out of it. Long Short-Term Memory networks, or LSTMs, are just a special type of RNN that can perform better when learning about “long-term dependencies”.

For example, if you have a large dataset of text you can train an LSTM model that will be able to learn the statistical structure of the text data. You can then sample from that model and create sequences of meanigul characters that will look like the original training data. So in other words, if you are trying to predict the last word in the following sentence:

We will use the LSTM generative capacity to create an interactive online demo where you can sample characters from a trained model and generate new sequences of text based on what you write.

or with Python:

Once you have created a Paperspace account you will be able to login in with your credentials from your command line:

Add your Paperspace email and password when prompted.

This will be the root of our project.

LSTMs work well when you want to predict sequences or patterns from a large dataset. Try to gather as much clean text data as you can! The more the better.

Once you have your data ready, create a new folder inside /data and called it anyway you want. Inside that new folder just add one file called input.txt that contains all your training data.

(A quick tip to concatenate many small disparate .txt files into one large training file: ls *.txt | xargs -L 1 cat >> input.txt)

Now we can start the training process. Just type:

This means we want to create a new paperspace job using as a base container a Docker image that comes with Tensorflow 1.5.1 and Python 3 installed (this way we don't need to worry about installing dependencies, packages or managing version). We also want to use a machineType P5000 and we want to run the command bash run.sh to start the training process. This project will be called LSTM training

If you typed this right (or copied it correctly), the training process should start and you should see something like this:

This might take take a while to run, LSTMs are known for talking time to train. A good thing is that you don’t need to monitor the complete process, but you can check how it is going by typing:

If you login into your Paperspace accout you can also follow the training process, under the Gradient tab, more interactively:

Once the training process is completed you should see the following log:

This will download all the files we will need that containin your trained model.

Now open the sketch.js file and change the name of your model in the following line:

The rest of the code is fairly straight forward. Once we create our lstm method with ml5js, we can make it sample the model by using the following function:

We are almost ready to test the model. The only thing left is to start a server to view our files. If you are using Python 2:

If you are using Python 3:

2 MB:

5–8 MB:

10–20 MB:

25+ MB:

Related posts:

In a surprisingly swift turn of events, the emergence of ChatGPT captivated the public’s imagination, sparking visions of a future filled with boundless possibilities. This remarkable AI assistant…

PreSeries predictive algorithms crawl the web hungry for startup information. So far, almost 400k companies have been ruthlessly processed, scored, and ranked. Today, we offer you a sneak peek at the…

When I first encountered Roger Williams, it was because of his prolific writings and his proximity in time, place and belief to one of my ancestors. Soon his story fascinated me. The Puritans in…