How To Get Automating AWS Server Shut-Down for Deep Learning

After reading that inshallah you will Automate AWS Server Shut-Down.

Lately, I have been working with an AWS EC2 server for a machine learning project. Every article I have come across has said the same thing… “Don’t forget to shutdown your EC2 instance”. Reading this gives me anxiety. Just the thought of leaving a GPU on to burn a hole in my pocket makes me irrationally sweaty.

So I decided to make a Bash script to spin up a server, SSH in so I can work on my project, then shut down the server when I log out. I even saved a shortcut for the script on my Desktop so that I can start working with two clicks.

Unfortunately, the process to get the script working was not straightforward and mildly infuriating. So I decided to write a quick post so the community can have the same peace of mind.

*My workstation is a Windows 10 PC, so these steps may be different for a Mac.*

Prerequisite 1: Have an AWS Account and Create an Instance

You will need to have an AWS account as well as have instantiated an AWS EC2 instance. FastAI has a tutorial on how to do this here. If this is your first time creating a GPU instance, you may have to request a limit increase for “p-type” instances. For me, this took about two days to hear back from AWS.

Prerequisite 2: Setting up the AWS CLI

You will need the AWS CLI so the Bash script can make calls to spin-up/shut-down the server. You can download the CLI from here.

AWS CLI Download

In the next few steps, we will create a User for yourself in AWS IAM and then generate and save security credentials so we can use the AWS CLI.

IAM Console

If you have a User in your IAM console, you do not need a new one. Just make sure that you have your user’s “security credentials.”

If you don’t have a User, create one.

If you create a user, make sure to give it “Programmatic Access”. Next, you will generate security keys for the user.

Save these keys!

In the Windows command prompt, you will need to run aws configure and follow the prompts. Fill in the Access Key ID and Secret Access Key with the key you got from the IAM console. Set your default region to the region your server is located in.

With these steps completed, you should be able to access your AWS instances from the command line. Try running aws ec2 describe-instances, you should see a list of your EC2 instances.

Prerequisite 3: Connecting Git-Bash to AWS

If you haven’t already, install Git-Bash from here. You will need this to run the Bash Script.

Git-Bash Download

The next step is to connect Git-Bash to the AWS CLI. To do this, you need to create a .bashrc file in the root directory of bash. Open up a Git-Bash terminal and do the following.

cd ~
vim .bashrc

This will open up a text editor. Where you can paste the following.

#!/bin/bash
alias aws=”winpty -Xallow-non-tty -Xplain C://Program\ Files/Amazon/AWSCLI/bin/aws.exe”

Now the Git-Bash terminal will recognize the command aws. If you skip this step, you will get the error bash: aws: command not found. Make sure that the file-path inserted above directs to aws.exe in your file directory.

Once completed, exit the vim terminal by typing esc then:wq then hit enter. For the changes to take effect, you will need to close and re-open Git-Bash. Do this and execute aws . You should see the following.

Testing the AWS CLI command

Creating the Script

With that, we are very close to our goal. Making a Bash script that automates spinning-up, logging into, and shutting down an EC2 server.

Now we can finish up and write that script. In your bash terminal navigate to wherever you want the script (for me cd C://Users/harrison/Desktop ) then run vim aws.sh

#!/bin/bash
INSTANCE_ID=<YOUR EC2 INSTANCE-ID HERE>
SSH_KEY=<path/to/sshkey.pem>
echo "Starting AWS Server."
aws ec2 start-instances --instance-ids $INSTANCE_ID

#Wait for server to come online
STARTED=False
while [ "$STARTED" != "True" ]; do
 SERVER_STATUS=$(aws ec2 describe-instance-status --include-all-instances --instance-ids $INSTANCE_ID --output text --query 'InstanceStatuses[].InstanceState.Name')
 if [ "$SERVER_STATUS" == "running" ]; then
    STARTED=True
 else
    sleep 3
 fi
 done
SERVER_PUBLIC_DNS=$(aws ec2 describe-instances — instance-ids $INSTANCE_ID — output text — query ‘Reservations[].Instances[].PublicDnsName’)
echo “Attempting to log into server.”
ssh -L localhost:8888:localhost:8888 -i $SSH_KEY ubuntu@$SERVER_PUBLIC_DNS
echo “Shutting down server.”
aws ec2 stop-instances — instance-ids $INSTANCE_ID
echo "Sent shutdown command."
sleep 1
echo "Querying instance status..."
aws ec2 describe-instance-status --include-all-instances --instance-ids $INSTANCE_ID --output text --query 'InstanceStatuses[]'
sleep 5

*Note: I will periodically make improvements to this script.*

Fill in INSTANCE_ID with the instance-id of your EC2 instance. Fill in SSH_KEY with the path to your SSH key.

Now when you want to start working in your EC2 server, just run the bash script and you will be connected to your server. From there, you can run a jupyter notebook or load in python scripts from GitHub. When you exit out of the instance, the script will continue running and turn-off the server for you.

How it works (Optional)

aws ec2 start-instances — instance-ids $INSTANCE_ID starts the EC2 instance.

You need the server’s public DNS to log-in. Unfortunately, the DNS can change when you turn the server off. The below code queries for the server’s public DNS.

SERVER_PUBLIC_DNS=$(aws ec2 describe-instances — instance-ids $INSTANCE_ID — output text — query ‘Reservations[].Instances[].PublicDnsName’)

Log in to the EC2 instance with ssh -L localhost:8888:localhost:8888 -i $SSH_KEY ubuntu@$SERVER_PUBLIC_DNS

aws ec2 stop-instances — instance-ids $INSTANCE_ID stops the EC2 instance.