Getting Started with AWS SDK for Java (1)

By , 2015年6月3日 10:48 上午

This is an entry level tutorial on how to use the AWS SDK for Java to interact with the various AWS services. Although we will cover a little bit about Java programming basics but this is not a tutorial on the Java programming language itself. If you want to learn the Java programming language language, I strongly recommend that you go through The Java Tutorial which was developed by Sun Microsystems in the very early days (improved and refined by Oracle later on).

To avoid exposing your AWS credentials in your code, all the examples in this tutorial use the credentials from IAM roles to authenticate with the AWS services. To run these examples, you will need to launch an EC2 instance (in this tutorial, we use Ubuntu 14.04 as the testing environment) with an IAM role. The IAM role should have sufficient permission to access the various AWS services you would like to test. For more information on this topic, please refer to the AWS documentation on IAM Roles for EC2.

[Java 8 SDK, AWS SDK for Java, Demo Code]

Assuming that you have launched an EC2 instance with a Ubuntu AMI, let’s SSH into the EC2 instance and install the Java 8 SDK:

$ sudo add-apt-repository ppa:webupd8team/java
$ sudo apt-get update
$ sudo apt-get install oracle-java8-installer

$ javac -version
javac 1.8.0_45
$ java -version
java version "1.8.0_45"
Java(TM) SE Runtime Environment (build 1.8.0_45-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.45-b02, mixed mode)

Now let’s download and configure the AWS SDK for Java:

$ cd ~
$ wget http://sdk-for-java.amazonwebservices.com/latest/aws-java-sdk.zip
$ sudo apt-get install unzip
$ unzip aws-java-sdk.zip

Now you have a folder with the name aws-java-sdk-1.x.xx in your home folder. The AWS libraries reside in the lib sub-folder while third-party dependencies reside in the third-party sub-folder. In order to use these libraries, you will need to configure your CLASSPATH to point to these sub-folders one by one. However, for the convenience of this tutorial, we simply copy everything into our JRE’s lib/ext sub-folder. With this approach, you don’t need to worry about the CLASSPATH at all. (If you are using other versions of the Java SDK, you will need to replace the following commands with the actual location of your Java installation.)

$ sudo cp aws-java-sdk-*/lib/*.jar /usr/lib/jvm/java-8-oracle/jre/lib/ext/
$ sudo cp aws-java-sdk-*/third-party/*/*.jar /usr/lib/jvm/java-8-oracle/jre/lib/ext/

The demo code for this tutorial is available in my github repository. The demo project uses maven as the project management code. So, we will need to install maven and git, then clone the code from my github repository.

$ cd ~
$ sudo apt-get install maven git
$ git clone https://github.com/qyjohn/aws-sdk-java-demo

Now we try to build the project and run a test application:

$ cd aws-sdk-java-demo
$ mvn compile
$ mvn package
$ java -cp target/demo-1.0-SNAPSHOT.jar net.qyjohn.aws.App
Hello World!

At this point, you have successfully configured your development environment and are ready to move forward to the rest of this tutorial.

[Amazon EC2 Client]

In this section, we use the AmazonEC2Client to accomplish some basic tasks such as launching an EC2 instance, listing all EC2 instances in a particular region, as well as terminating a particular EC2 instance. The related source code for this demo is DemoEC2.java (you can click on the link to view the source code in a separate browser tab). You should also take a look at the Java docs for the AmazonEC2Client to get yourself familiar with the various properties and methods.

We create an instance of the AmazonEC2Client in the constructor. At the same time, we specify which region we are going to use. You should always specify a region when manipulation AWS resources, unless the resource you are operating on is global (for example, IAM).

public class DemoEC2 
{
	public AmazonEC2Client client;

	/**
	 *
	 * Constructor
	 *
	 */

	public DemoEC2()
	{
		// Create the AmazonEC2Client
		client = new AmazonEC2Client();
		// Set the region to ap-southeast-2
		client.setRegion(Regions.AP_SOUTHEAST_2);
	}

To launch an EC2 instance, you will need to create a StartInstancesRequest object,  then pass it to the startInstances() method of the AmazonEC2Client, which returns a StartInstancesResult object. As shown in the following demo code, the StartInstancesRequest object contains information such as the AMI, the instance type, the key pair, the subnet, the security group, the number of instances to be launched. Many many other options can be supplied to the StartInstancesRequest object. You will need to refer to the API docs when needed.

	public String launchInstance()
	{
		System.out.println("\n\nLAUNCH INSTANCE\n\n");

		try
		{
			// Construct a RunInstancesRequest.
			RunInstancesRequest request = new RunInstancesRequest();
			request.setImageId("ami-fd9cecc7");	// the AMI ID, ami-fd9cecc7 is Amazon Linux AMI 2015.03 (HVM)
			request.setInstanceType("t2.micro");	// instance type
			request.setKeyName("desktop");		// the keypair
			request.setSubnetId("subnet-2dc0d459");	// the subnet
			ArrayList list = new ArrayList();
			list.add("sg-efcc248a");			// security group, call add() again to add more than one
			request.setSecurityGroupIds(list);
			request.setMinCount(1);	// minimum number of instances to be launched
			request.setMaxCount(1);	// maximum number of instances to be launched

			// Pass the RunInstancesRequest to EC2.
			RunInstancesResult  result  = client.runInstances(request);
			String instanceId = result.getReservation().getInstances().get(0).getInstanceId();
			
			// Return the first instance id in this reservation.
			// So, don't launch multiple instances with this demo code.
			System.out.println("Launching instance " + instanceId);
			return instanceId;
		} catch (Exception e)
		{
			// Simple exception handling by printing out error message and stack trace
			System.out.println(e.getMessage());
			e.printStackTrace();
			return "ERROR";
		}
	}

To list all the EC2 instances in a region, we simply call the describeInstances() method with no argument, which returns a DescribeInstancesResult. In the DescribeInstancesResult, traverse through all the Reservation, which is stored in a List. Each Reservation includes one or more EC2 Instance, which is also stored in a List. You can get the information for each EC2 instance from the Instance object.

The concept of Reservation seems to be confusing, and many people mistakenly think that it is the same as Reserved Instances but in fact it is not. According to the boto documentation, a reservation corresponds to a command to start instances. If you launch two EC2 instance in one batch (for example, specifying the number of instances in the EC2 Console), this particular Reservation will have two EC2 instances. You can stop and started the EC2 instances to change their state, but this will not change their Reservation.

	public void listInstances()
	{
		System.out.println("\n\nLIST INSTANCE\n\n");
        	try 
		{
			// DescribeInstances
			DescribeInstancesResult result = client.describeInstances();

			// Traverse through the reservations
			List reservations = result.getReservations();
			for (Reservation reservation: reservations)
			{
				// Print out the reservation id
				String reservation_id = reservation.getReservationId();
				System.out.println("Reservation: " + reservation_id);
				// Traverse through the instances in a reservation
				List instances = reservation.getInstances();
				for (Instance instance: instances)
				{
					// Print out some information about the instance
					String id = instance.getInstanceId();
					String state = instance.getState().getName();
					System.out.println("\t" + id + "\t" + state);
				}
			}

	        } catch (Exception e) 
		{
			// Simple exception handling by printing out error message and stack trace
			System.out.println(e.getMessage());
			e.printStackTrace();
		}
	}

To terminate an EC2 instance, you will need to create a TerminateInstancesRequest object. The TerminateInstancesRequest object accepts a List of EC2 instance id through the setInstanceIds() method. Then you pass the TerminateInstancesRequest to the AmazonEC2Client’s terminateInstances() method, which returns a TerminateInstancesResult object. In the TerminateInstancesResult object, you have a List of InstanceStateChange, and each InstanceStateChange object contains information about the EC2 instance id, it’s previous state, and it’s current state.

	public void terminateInstance(String instanceId)
	{
		System.out.println("\n\nTERMINATE INSTANCE\n\n");
		try
		{
			// Construct the TerminateInstancesRequest
			TerminateInstancesRequest request = new TerminateInstancesRequest();
			ArrayList list = new ArrayList();
			list.add(instanceId);			// instance id
			request.setInstanceIds(list);

			// Pass the TerminateInstancesRequest to EC2
			TerminateInstancesResult result = client.terminateInstances(request);
			List  changes = result.getTerminatingInstances();
			for (InstanceStateChange change : changes)
			{
				String id = change.getInstanceId();
				String state_prev = change.getPreviousState().toString();
				String state_next = change.getCurrentState().toString();
				System.out.println("Instance " + id + " is changing from " + state_prev + " to " + state_next + ".");
			}
		} catch (Exception e)
		{
			// Simple exception handling by printing out error message and stack trace
			System.out.println(e.getMessage());
			e.printStackTrace();
		}
	}

Now you can make modifications to the demo code (in DemoEC2.java) with your preferred AWS region, AMI, instance type, along with other information. You can run the demo code using the following commands:

$ mvn compile
$ mvn package
$ java -cp target/demo-1.0-SNAPSHOT.jar net.qyjohn.aws.DemoEC2

This demo will launch one EC2 instance, list all the EC2 instances in the region, terminate the EC2 instance we just launched, and list all the EC2 instances in the region again. In order to demonstrate the state changes, we do a sleep of 10 seconds between each API call. After you have completed this exercise, you should intentionally introduce some errors in the code to observe the various exceptions thrown by the demo code. For example, you can point your AmazonEC2Client to use the us-east-1 region, but specify a subnet id or a security group id in another region.

[Logging Considerations]

When you encountered an error when making an API call using the AWS SDK for Java, the chance is that you have some mistake in the AWS resource you specified in the code, while the code itself is completely valid. In order to debug such issues, you will need to know what information is being sent to the AWS endpoint, and what information is returned from the AWs endpoint. In the dark age, debugging an application can only be achieved by looking into the Java object containing the request and result, then print out the information one by one using System.out.println(). Fortunately you don’t need to do this with the AWS SDK for Java, because the AWS SDK for Java is instrumented with Apache Commons Logging. All you need to do is grab a recent copy of log4j and set up the proper CLASSPATH, then ask your application to use the proper log4j configuration file.

In this demo, we have the JAR files for log4j version 2.3 in the third-party folder. we simply copy everything into our JRE’s lib/ext sub-folder. With this approach, you don’t need to worry about the CLASSPATH at all.

$ cd third-party
$ sudo cp *.jar /usr/lib/jvm/java-8-oracle/jre/lib/ext/

The log4j configuration file log4j2.xml, which is located at the top level folder of the demo code. You should refer to the log4j manual to understand how log4j works. To enable log4j for our demo application, you simply need to uncomment the Logger line in DemoEC2.java, as below:

public class DemoEC2 
{
	public AmazonEC2Client client;
	final static Logger logger = Logger.getLogger(DemoEC2.class);

After that, you will need to compile and package the demo code again. When running the Java application, you will need to pass the log4j configuration file to the java command:

$ mvn compile
$ mvn package
$ java -cp target/demo-1.0-SNAPSHOT.jar -Dlog4j.configurationFile=log4j2.xml net.qyjohn.aws.DemoEC2

As you can see, when you run the application, the request you send to the AWS endpoint, as well as the response from the AWS endpoint, are now display on your screen. With this information, it is a lot easier to debug your API calls to AWS endpoints.

By now we have completed the first chapter of this “Getting Started with AWS SDK for Java” tutorial. In the future I will publish more on this topic on a irregular base. Hopefully I will be able to cover the majority of AWS services that we use on a daily base. So, please stay tuned for my future updates.

2 Responses to “Getting Started with AWS SDK for Java (1)”

  1. […] already do so, I suggest that you first take a look at the first chapter of this set of training “Getting Started with AWS SDK for Java (1)” to properly set up your development environment. In this part, we will cover the Amazon RDS […]

  2. qyjohn说道:

    With AWS SDK for Java version 1.9.38, the following Syntax worked:

    // Set the region to ap-southeast-2
    client.setRegion(Regions.AP_SOUTHEAST_2);

    However, starting from version 1.9.39, this will no longer work. Instead, you will need to do the following:

    // Set the region to ap-southeast-2
    client.configureRegion(Regions.AP_SOUTHEAST_2);

    or

    // Set the region to ap-southeast-2
    client.setEndpoint(“ec2.ap-southeast-2.amazonaws.com”);

Leave a Reply

Panorama Theme by Themocracy