I was able to install CUDA 8 on the EC2 instance with the following steps. It should be noted that the EC2 instance was created with a root EBS volume of 100 GB to avoid running into storage space issues.

#
# STEP 1: Install Nvidia Driver
# 384.66 is a version that has support for K80
#
cd ~
sudo yum install -y gcc kernel-devel-`uname -r`
wget http://us.download.nvidia.com/XFree86/Linux-x86_64/384.66/NVIDIA-Linux-x86_64-384.66.run
sudo /bin/bash ./NVIDIA-Linux-x86_64-384.66.run
nvidia-smi

#
# STEP 2: Install CUDA Repo
#
wget https://developer.nvidia.com/compute/cuda/8.0/Prod2/local_installers/cuda-repo-rhel6-8-0-local-ga2-8.0.61-1.x86_64-rpm
sudo rpm -i cuda-repo-rhel6-8-0-local-ga2-8.0.61-1.x86_64-rpm

#
# STEP 3: Install CUDA Toolkit
#
sudo yum install cuda-toolkit-8-0
export PATH=$PATH:/usr/local/cuda-8.0/bin
nvcc –version

#
# STEP 4: Compile a sample program (deviceQuery) to use CUDA
#
cd /usr/local/cuda-8.0
sudo chown -R ec2-user:ec2-user samples
cd samples/1_Utilities/deviceQuery
make
./deviceQuery

At this point everything should be all set. I have also compiled and tested some other sample code from the samples folder and they all seemed to work.

A quick example on cuBLAS can be obtained from http://docs.nvidia.com/cuda/cublas/ . Simply copy Example 1 or Example 2 from this web page and save it as test.c, then compile and run the code with the following commands. I have tested both of them and verified them to be working.

#
# STEP 5: Compile and test cuBLAS code
#
nvcc test.c -lcublas -o test
./test