Getting Started with AWS SDK for Java (2)

By , June 11, 2015 10:24 am

This is the 2nd part of my tutorial on “Getting Started with AWS SDK for Java”. If you have not already do so, I suggest that you first take a look at the first chapter of this set of training “Getting Started with AWS SDK for Java (1)” to properly set up your development environment. In this part, we will cover the Amazon RDS client, as well as some common issues when using RDS as the back end database for your Java applications.

[Amazon RDS Client]

In this section, we use the AmazonRDSClient to accomplish some basic tasks such as launching an RDS instance, listing all RDS instances in a particular region, as well as terminating a particular RDS instance. The related source code for this demo is DemoRDS.java (you can click on the link to view the source code in a separate browser tab). You should also take a look at the Java docs for the AmazonRDSClient to get yourself familiar with the various properties and methods.

First of all we create an instance of the AmazonRDSClient in the constructor, then set the region to ap-southeast-2. For debugging purposes, we enable logging using log4j.

	
public class DemoRDS 
{
	public AmazonRDSClient client;
	final static Logger logger = Logger.getLogger(DemoRDS.class);

	/**
	 *
	 * Constructor
	 *
	 */

	public DemoRDS()
	{
		// Create the AmazonRDSClient
		client = new AmazonRDSClient();
		// Set the region to ap-southeast-2
		client.setRegion(Regions.AP_SOUTHEAST_2);
	}

To launch an RDS instance, you will need to create a CreateDBInstanceRequest object, then pass it to the createDBInstance() method of the AmazonRDSClient, which returns a DBInstance object. From the DBInstance object, you will be able to obtain information about the newly created RDS instance. Due to the asynchronous nature of AWS API calls, some information might not be available in the DBInstance object returned by the createDBInstance() method. For example, the DNS endpoint for the newly created RDS instance will not be available until several minutes later, therefore instance.getEndpoint() will return a null result. If you try to convert this null result into a String, you will get an exception.

	public String launchInstance()
	{
		System.out.println("\n\nLAUNCH INSTANCE\n\n");

		try
		{
			// The CreateDBInstanceRequest object
			CreateDBInstanceRequest request = new CreateDBInstanceRequest();
			request.setDBInstanceIdentifier("Sydney");	// RDS instance name
			request.setDBInstanceClass("db.t2.micro");
			request.setEngine("MySQL");		
			request.setMultiAZ(false);
			request.setMasterUsername("username");
			request.setMasterUserPassword("password");
			request.setDBName("mydb");		// database name 
			request.setStorageType("gp2");		// standard, gp2, io1
			request.setAllocatedStorage(10);	// in GB

			// VPC security groups 
			ArrayList list = new ArrayList();
			list.add("sg-efcc248a");			// security group, call add() again to add more than one
			request.setVpcSecurityGroupIds(list);

			// Create the RDS instance
			DBInstance instance = client.createDBInstance(request);

			// Information about the new RDS instance
			String identifier = instance.getDBInstanceIdentifier();
			String status = instance.getDBInstanceStatus();
			Endpoint endpoint = instance.getEndpoint();
			String endpoint_url = "Endpoint URL not available yet.";
			if (endpoint != null)
			{
				endpoint_url = endpoint.toString();
			}

			// Do some printing work
			System.out.println(identifier + "\t" + status);
			System.out.println(endpoint_url);

			// Return the DB instance identifier
			return identifier;
		} catch (Exception e)
		{
			// Simple exception handling by printing out error message and stack trace
			System.out.println(e.getMessage());
			e.printStackTrace();
			return "ERROR";
		}
	}

To list all RDS instances, we simply call the describeDBInstances() method of the AmazonRDSClient. This method returns a list of DBInstance objects, and you need to traverse through the list to obtain information about each individual DBInstance object.

	public void listInstances()
	{
		System.out.println("\n\nLIST INSTANCE\n\n");
        	try 
		{
			// Describe DB instances
			DescribeDBInstancesResult result = client.describeDBInstances();
			
			// Getting a list of the RDS instances
			List instances = result.getDBInstances();
			for (DBInstance instance : instances)
			{
				// Information about each RDS instance
				String identifier = instance.getDBInstanceIdentifier();
				String engine = instance.getEngine();
				String status = instance.getDBInstanceStatus();
				Endpoint endpoint = instance.getEndpoint();
				String endpoint_url = "Endpoint URL not available yet.";
				if (endpoint != null)
				{
					endpoint_url = endpoint.toString();
				}

				// Do some printing work
				System.out.println(identifier + "\t" + engine + "\t" + status);
				System.out.println("\t" + endpoint_url);
			}
	        } catch (Exception e) 
		{
			// Simple exception handling by printing out error message and stack trace
			System.out.println(e.getMessage());
			e.printStackTrace();
		}
	}

To terminate an RDS instance, we need to create a DeleteDBInstanceRequest, then pass the DeleteDBInstanceRequest to the deleteDBInstance() method. In the DeleteDBInstanceRequest, you should at least specify the DB instance identifier and whether you want to skip the final snapshot for the RDS instance to be deleted. If you want to create a final snapshot, you will need to set the name of the final snapshot in the DeleteDBInstanceRequest object.

	public void terminateInstance(String identifier)
	{
		System.out.println("\n\nTERMINATE INSTANCE\n\n");
		try
		{
			// The DeleteDBInstanceRequest 
			DeleteDBInstanceRequest request = new DeleteDBInstanceRequest();
			request.setDBInstanceIdentifier(identifier);
			request.setSkipFinalSnapshot(true);
			
			// Delete the RDS instance
			DBInstance instance = client.deleteDBInstance(request);

			// Information about the RDS instance being deleted
			String status = instance.getDBInstanceStatus();
			Endpoint endpoint = instance.getEndpoint();
			String endpoint_url = "Endpoint URL not available yet.";
			if (endpoint != null)
			{
				endpoint_url = endpoint.toString();
			}

			// Do some printing work
			System.out.println(identifier + "\t" + status);
			System.out.println(endpoint_url);
		} catch (Exception e)
		{
			// Simple exception handling by printing out error message and stack trace
			System.out.println(e.getMessage());
			e.printStackTrace();
		}
	}

Before running the demo code, please modify the source code with the appropriate arguments (such as the security groups when creating the RDS instance) for the API calls. It is recommended that you intentionally introduce some errors in the arguments to observe the logging information from the AWS SDK. The demo code comes with switches for each demo module. You can use the launch, list, and terminate switches to pick which demo module you would like to run. For example:

$ mvn compile
$ mvn package
$ java -cp target/demo-1.0-SNAPSHOT.jar -Dlog4j.configurationFile=log4j2.xml net.qyjohn.aws.DemoRDS launch
$ java -cp target/demo-1.0-SNAPSHOT.jar -Dlog4j.configurationFile=log4j2.xml net.qyjohn.aws.DemoRDS list
$ java -cp target/demo-1.0-SNAPSHOT.jar -Dlog4j.configurationFile=log4j2.xml net.qyjohn.aws.DemoRDS list
$ java -cp target/demo-1.0-SNAPSHOT.jar -Dlog4j.configurationFile=log4j2.xml net.qyjohn.aws.DemoRDS terminate
$ java -cp target/demo-1.0-SNAPSHOT.jar -Dlog4j.configurationFile=log4j2.xml net.qyjohn.aws.DemoRDS list
$ java -cp target/demo-1.0-SNAPSHOT.jar -Dlog4j.configurationFile=log4j2.xml net.qyjohn.aws.DemoRDS list

[JDBC Basics]

With Java, people interact with database using JDBC (Java Database Connectivity). This is done in a 4-step approach:

- loading the JDBC driver using a class loader
- establishing a connection using DriverManager
- working with the database
- close the connection

The JDBC drivers for MySQL, PostgreSQL, Oracle and SQL Server can be found from the following URL. You will need to put the corresponding JAR file into your CLASSPATH to make things work. In the third-party folder of our demo code, we provide a copy of MySQL Connector/J 5.1.35.

- MySQL Connector/J
- PostgreSQL JDBC Driver
- Oracle JDBC Driver
- Microsoft JDBC Driver for SQL Server

With JDBC, we connect to database using connection URL, which includes properties such as the hostname or IP address of the database server, the port number to use for the connection, the name of the database to work with, as well as username and password. For different database engines, the format of the connection URL is slightly different. The following pseudo-code provides example connection URLs for MySQL, PostgreSQL, Oracle and SQL Server. If you need a definitive guidance on constructing connection URL for a specific database engine, please refer to the following URL:

- JDBC Connection URL for MySQL
- JDBC Connection URL for PostgreSQL
- JDBC Connection URL for Oracle
- JDBC Connection URL for SQL Server

	// MySQL
	Class.forName("com.mysql.jdbc.Driver");
	String jdbc_url = "jdbc:mysql://hostname/database?user=username&password=password";
	Connection conn = DriverManager.getConnection(jdbc_url);
	
	// PostgreSQL
	Class.forName("org.postgresql.Driver");
	String jdbc_url = "jdbc:postgresql://hostname/database?user=username&password=password&ssl=true"";
	Connection conn = DriverManager.getConnection(jdbc_url);
	
	// Oracle
	Class.forName ("oracle.jdbc.OracleDriver");
	String jdbc_url = "jdbc:oracle:thin:@hostname:1521:orcl";
	Connection conn = DriverManager.getConnection(jdbc_url, "username", "password");	
	
	// SQL Server
	Class.forName("com.microsoft.jdbc.sqlserver.SQLServerDriver");
	String jdbc_url = "jdbc:microsoft:sqlserver://hostname:1433;DatabaseName=database";
	Connection conn = DriverManager.getConnection(jdbc_url, "username", "password");

The following demo code provides an example on using MySQL Connector/J to connect to an RDS instance, then carry out some operations such as CREATE TABLE, INSERT, and SELECT in an infinite loop. The properties of the database (including hostname, database, username, password) are provided in a property file db.properties in the top level folder of the demo code.  When we run the demo code, we load these properties from an InputStream. This way we do not need to provide database credentials in the source code. (The benefit of doing this is that when your database credentials changes, you do not need to recompile your Java code. All you need to do is to update the properties in db.properties.)

In this demo code we catch Exception in two levels – the first level Exception might occur when loading the property file (file does not exist, incorrect format, or required entry missing) or loading the MySQL JDBC driver (the JAR file is not in CLASSPATH), while the second level Exception might occur within the infinite loop (can not open a connection to the database, can not CREATE TABLE or execute INSERT or SELECT queries). When the first level Exception occurs, there are errors in the resource level, so we can’t move forward at all. When the second level Exception occurs, there might be things that we can fix from within the RDS instance, so we simply print out the error messages and keep on trying using the infinite loop.

	public void runJdbcTests()
	{
		System.out.println("\n\nJDBC TESTS\n\n");
		try 
		{
			// Getting database properties from db.properties
			Properties prop = new Properties();
			InputStream input = new FileInputStream("db.properties");
			prop.load(input);
			String db_hostname = prop.getProperty("db_hostname");
			String db_username = prop.getProperty("db_username");
			String db_password = prop.getProperty("db_password");
			String db_database = prop.getProperty("db_database");

			// Load the MySQL JDBC driver
			Class.forName("com.mysql.jdbc.Driver");
			String jdbc_url = "jdbc:mysql://" + db_hostname + "/" + db_database + "?user=" + db_username + "&password=" + db_password;

			// Run an infinite loop 
			Connection conn = null;
			while (true)
			{
				try
				{
					// Create a connection using the JDBC driver
					conn = DriverManager.getConnection(jdbc_url);

					// Create the test table if not exists
					Statement statement = conn.createStatement();
					String sql = "CREATE TABLE IF NOT EXISTS jdbc_test (id INT NOT NULL AUTO_INCREMENT PRIMARY KEY, content VARCHAR(80))";
					statement.executeUpdate(sql);

					// Do some INSERT
					PreparedStatement preparedStatement = conn.prepareStatement("INSERT INTO jdbc_test (content) VALUES (?)");
					String content = "" + UUID.randomUUID();
					preparedStatement.setString(1, content);
					preparedStatement.executeUpdate();
					System.out.println("INSERT: " + content);

					// Do some SELECT
					sql = "SELECT COUNT(*) as count FROM jdbc_test";
					ResultSet resultSet = statement.executeQuery(sql);
					if (resultSet.next())
					{
						int count = resultSet.getInt("count");
						System.out.println("Total Records: " + count);
					}

					// Close the connection
					conn.close();

					// Sleep for some time
					Thread.sleep(20000);
				} catch (Exception e1)
				{
					System.out.println(e1.getMessage());
					e1.printStackTrace();
				}
			}
		} catch (Exception e0)
		{
			System.out.println(e0.getMessage());
			e0.printStackTrace();
		}
	}

After creating an RDS instance and update the properties in db.properties, you can run the JDBC tests using the following command. You can stop the execution of this demo using CTRL C.

$ java -cp target/demo-1.0-SNAPSHOT.jar -Dlog4j.configurationFile=log4j2.xml net.qyjohn.aws.DemoRDS jdbc

JDBC TESTS

INSERT: cc6294da-fb84-4c6f-aa49-a33804058d03
Total Records: 1
INSERT: 1d7c8940-79cc-45ca-948a-27b809bb9e69
Total Records: 2
INSERT: 32d1acc5-c9ed-4bce-a6cd-44e7ac38ff42
Total Records: 3
INSERT: 88923f13-5ecd-41c5-a437-2d51099c2ff5
Total Records: 4

[Cloud Specific Considerations]

When building applications on top of AWS, it is important to assume that everything fails all the time. With this in mind, you should always connect to your RDS instance using the DNS endpoint instead of the IP address obtained from a DNS server, because the IP address of your RDS instance will change when a Multi-AZ fail over or a Single-AZ recovery occurs. In the case of Multi-AZ fail over, the DNS endpoint will be resolved to the IP address of the new master. In the case of Single-AZ recovery, a new instance will be launched and the DNS endpoint will be resolved to the IP address of the new instance.

For example, if we do a “reboot with fail over” of the RDS instance while running our JDBC tests against the RDS instance, we will see the following output. An Exception occurs when the Multi-AZ fail over occurs because the JDBC driver fails to connect to the old master due to connection timeout. When the Multi-AZ fail over is completed, subsequent connections are make to the new master successfully. (This is why we put each set of test inside a try… catch block.)

$ java -cp target/demo-1.0-SNAPSHOT.jar -Dlog4j.configurationFile=log4j2.xml -Djava.security.manager=default net.qyjohn.aws.DemoRDS jdbc

JDBC TESTS

INSERT: 14a03563-325e-4dc3-8456-bdbc1fee3034
Total Records: 48
INSERT: ec28a659-fc64-4995-b434-760e8b3274ae
Total Records: 49
Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

INSERT: d59f3e77-1a17-4bee-9056-018dde60fb27
Total Records: 50
INSERT: 4ef1b7c7-1de1-430f-a503-61227c4b1249
Total Records: 51

In the above-mentioned demo, the Java SE application successfully handle the Multi-AZ event. However, this might not be the case in a Java EE use case, where a security manger is in place. The reason is that in Java there is a networking property networkaddress.cache.ttl controlling the caching policy for successful name lookups from the name service (see Java Properties for details). A value of -1 indicates “cache forever”. The default behavior is to cache forever when a security manager is installed (with Java EE applications, this is a common practice enforced by the application server). When a Multi-AZ fail over is completed, the operating system already sees the new DNS record for the RDS endpoint, but the Java application still keeps the old DNS record. The result is, when you have a Java EE application running in Tomcat, JBoss, or GlassFish, the application keeps on trying to reach the old master (which is no longer in service) and keeps on failing, until a restart of the application.

We can simulate this behavior with the same JDBC tests. Before doing this, we need to add the following security manager entry to /usr/lib/jvm/java-8-oracle/jre/lib/security/java.policy (You should replace /home/ubuntu/aws-sdk-java-demo with the actual path of your demo code folder). This policy grants AllPermission to our demo application, which is represented by the JAR in the target folder.

grant codeBase "file:/home/ubuntu/aws-sdk-java-demo/target/*"
{
	permission java.security.AllPermission;
};

Then we run our demo application again with the default security manager, then do another “reboot with fail over” of the RDS instance while running our JDBC tests. Now we should see that the application fails to open a connection to the RDS instance for ever. If you do a “dig” against the DNS endpoint of the RDS instance before and after the fail over, you will see that the operating system does see the change in DNS records.

$ java -cp target/demo-1.0-SNAPSHOT.jar -Dlog4j.configurationFile=log4j2.xml -Djava.security.manager=default net.qyjohn.aws.DemoRDS jdbc

JDBC TESTS

INSERT: b9b8cb31-5ef7-41ed-a1d6-96f430bb6cb2
Total Records: 52
INSERT: 9afa8447-0876-4af5-88ce-e2aa782f91f8
Total Records: 53
Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

There are many different solutions to this issue, including (1) modifying the required system property in the java command line; (2) modifying the /usr/lib/jvm/java-8-oracle/jre/lib/security/java.security configuration file; (3) modifying the startup parameters of your application server; and (4) setting up a new value for networkaddress.cache.ttl directly in your Java code.

The first solution is to add a Dsun.net.inetaddr.ttl=0 (never cache) to your command line. As shown in the following example, with this setting our JDBC test is able to pick up the new DNS record after one Exception.

$ java -cp target/demo-1.0-SNAPSHOT.jar -Dlog4j.configurationFile=log4j2.xml -Djava.security.manager=default -Dsun.net.inetaddr.ttl=0 net.qyjohn.aws.DemoRDS jdbc

JDBC TESTS

INSERT: a8c1bcca-0335-4217-9f97-a3964f12c574
Total Records: 54
INSERT: 5fa1cb23-96b9-449f-98b8-b0184781d657
Total Records: 55
Communications link failure

The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

INSERT: 4e3c8e51-9857-4024-8dc2-5a86f442a260
Total Records: 56
INSERT: 9ad6ec95-159a-4291-852d-903a08efc065
Total Records: 57

The second solution is to set a value for the networkaddress.cache.ttl property (0 for never cache) permanently in /usr/lib/jvm/java-8-oracle/jre/lib/security/java.security. This can be done by adding the following one line to this configuration file:

networkaddress.cache.ttl=0

The third solution is to modify the startup parameters of your application server. In the case of Tomcat7, you can modify JAVA_OPTS in /etc/default/tomcat7 with the desired setting, as below:

# You may pass JVM startup parameters to Java here. If unset, the default
# options will be: -Djava.awt.headless=true -Xmx128m -XX:+UseConcMarkSweepGC
#
# Use "-XX:+UseConcMarkSweepGC" to enable the CMS garbage collector (improved
# response time). If you use that option and you run Tomcat on a machine with
# exactly one CPU chip that contains one or two cores, you should also add
# the "-XX:+CMSIncrementalMode" option.
JAVA_OPTS="-Djava.awt.headless=true -Dsun.net.inetaddr.ttl=0 -Xmx128m -XX:+UseConcMarkSweepGC"

The fourth solution is to set up a new value for networkaddress.cache.ttl directly in your Java code, as describe by this AWS documentation Setting the JVM TTL for DNS Name Lookups. If you have the ability to modify your code, this is the recommended way, because you have full control of the behavior of your application, regardless of the configuration of the underlying runtime environment. (In the example below, 60 indicates the new TTL is 60 seconds. This way you still have some caching, but the caching is not that aggressive.)

java.security.Security.setProperty("networkaddress.cache.ttl" , "60");

The above-mentioned “Communication link failure” is one of the most commonly seen errors when working with RDS MySQL instances. In most cases, this issue can be resolved by asking yourself the following questions:

- Does your security group allows the communication between your EC2 instance (or on premise server) to communicate with your RDS instance (do a telnet to port 3306 on the RDS instance for a quick test)?

- Is a connection pool being used? Do you validate the connection when checking it out from the connection pool? Existing connections in a connection pool might become invalid due to various reasons (for example, timeouts).

- Has the MySQL service daemon been restarted? With RDS, the MySQL service daemon automatically restarts after it is crashed due to various reasons (for example, out of memory errors).

- Is there a fail over event (Multi-AZ) or recovery event (Single-AZ)?

- Does the operating system has the correct DNS record (do a dig to verify)? Does your Java application has the correct DNS record (check networkaddress.cache.ttl)?

- Is there anything in MySQL error log?

Getting Started with AWS SDK for Java (1)

By , June 3, 2015 10:48 am

This is an entry level tutorial on how to use the AWS SDK for Java to interact with the various AWS services. Although we will cover a little bit about Java programming basics but this is not a tutorial on the Java programming language itself. If you want to learn the Java programming language language, I strongly recommend that you go through The Java Tutorial which was developed by Sun Microsystems in the very early days (improved and refined by Oracle later on).

To avoid exposing your AWS credentials in your code, all the examples in this tutorial use the credentials from IAM roles to authenticate with the AWS services. To run these examples, you will need to launch an EC2 instance (in this tutorial, we use Ubuntu 14.04 as the testing environment) with an IAM role. The IAM role should have sufficient permission to access the various AWS services you would like to test. For more information on this topic, please refer to the AWS documentation on IAM Roles for EC2.

[Java 8 SDK, AWS SDK for Java, Demo Code]

Assuming that you have launched an EC2 instance with a Ubuntu AMI, let’s SSH into the EC2 instance and install the Java 8 SDK:

$ sudo add-apt-repository ppa:webupd8team/java
$ sudo apt-get update
$ sudo apt-get install oracle-java8-installer

$ javac -version
javac 1.8.0_45
$ java -version
java version "1.8.0_45"
Java(TM) SE Runtime Environment (build 1.8.0_45-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.45-b02, mixed mode)

Now let’s download and configure the AWS SDK for Java:

$ cd ~
$ wget http://sdk-for-java.amazonwebservices.com/latest/aws-java-sdk.zip
$ sudo apt-get install unzip
$ unzip aws-java-sdk.zip

Now you have a folder with the name aws-java-sdk-1.x.xx in your home folder. The AWS libraries reside in the lib sub-folder while third-party dependencies reside in the third-party sub-folder. In order to use these libraries, you will need to configure your CLASSPATH to point to these sub-folders one by one. However, for the convenience of this tutorial, we simply copy everything into our JRE’s lib/ext sub-folder. With this approach, you don’t need to worry about the CLASSPATH at all. (If you are using other versions of the Java SDK, you will need to replace the following commands with the actual location of your Java installation.)

$ sudo cp aws-java-sdk-*/lib/*.jar /usr/lib/jvm/java-8-oracle/jre/lib/ext/
$ sudo cp aws-java-sdk-*/third-party/*/*.jar /usr/lib/jvm/java-8-oracle/jre/lib/ext/

The demo code for this tutorial is available in my github repository. The demo project uses maven as the project management code. So, we will need to install maven and git, then clone the code from my github repository.

$ cd ~
$ sudo apt-get install maven git
$ git clone https://github.com/qyjohn/aws-sdk-java-demo

Now we try to build the project and run a test application:

$ cd aws-sdk-java-demo
$ mvn compile
$ mvn package
$ java -cp target/demo-1.0-SNAPSHOT.jar net.qyjohn.aws.App
Hello World!

At this point, you have successfully configured your development environment and are ready to move forward to the rest of this tutorial.

[Amazon EC2 Client]

In this section, we use the AmazonEC2Client to accomplish some basic tasks such as launching an EC2 instance, listing all EC2 instances in a particular region, as well as terminating a particular EC2 instance. The related source code for this demo is DemoEC2.java (you can click on the link to view the source code in a separate browser tab). You should also take a look at the Java docs for the AmazonEC2Client to get yourself familiar with the various properties and methods.

We create an instance of the AmazonEC2Client in the constructor. At the same time, we specify which region we are going to use. You should always specify a region when manipulation AWS resources, unless the resource you are operating on is global (for example, IAM).

public class DemoEC2 
{
	public AmazonEC2Client client;

	/**
	 *
	 * Constructor
	 *
	 */

	public DemoEC2()
	{
		// Create the AmazonEC2Client
		client = new AmazonEC2Client();
		// Set the region to ap-southeast-2
		client.setRegion(Regions.AP_SOUTHEAST_2);
	}

To launch an EC2 instance, you will need to create a StartInstancesRequest object,  then pass it to the startInstances() method of the AmazonEC2Client, which returns a StartInstancesResult object. As shown in the following demo code, the StartInstancesRequest object contains information such as the AMI, the instance type, the key pair, the subnet, the security group, the number of instances to be launched. Many many other options can be supplied to the StartInstancesRequest object. You will need to refer to the API docs when needed.

	public String launchInstance()
	{
		System.out.println("\n\nLAUNCH INSTANCE\n\n");

		try
		{
			// Construct a RunInstancesRequest.
			RunInstancesRequest request = new RunInstancesRequest();
			request.setImageId("ami-fd9cecc7");	// the AMI ID, ami-fd9cecc7 is Amazon Linux AMI 2015.03 (HVM)
			request.setInstanceType("t2.micro");	// instance type
			request.setKeyName("desktop");		// the keypair
			request.setSubnetId("subnet-2dc0d459");	// the subnet
			ArrayList list = new ArrayList();
			list.add("sg-efcc248a");			// security group, call add() again to add more than one
			request.setSecurityGroupIds(list);
			request.setMinCount(1);	// minimum number of instances to be launched
			request.setMaxCount(1);	// maximum number of instances to be launched

			// Pass the RunInstancesRequest to EC2.
			RunInstancesResult  result  = client.runInstances(request);
			String instanceId = result.getReservation().getInstances().get(0).getInstanceId();
			
			// Return the first instance id in this reservation.
			// So, don't launch multiple instances with this demo code.
			System.out.println("Launching instance " + instanceId);
			return instanceId;
		} catch (Exception e)
		{
			// Simple exception handling by printing out error message and stack trace
			System.out.println(e.getMessage());
			e.printStackTrace();
			return "ERROR";
		}
	}

To list all the EC2 instances in a region, we simply call the describeInstances() method with no argument, which returns a DescribeInstancesResult. In the DescribeInstancesResult, traverse through all the Reservation, which is stored in a List. Each Reservation includes one or more EC2 Instance, which is also stored in a List. You can get the information for each EC2 instance from the Instance object.

The concept of Reservation seems to be confusing, and many people mistakenly think that it is the same as Reserved Instances but in fact it is not. According to the boto documentation, a reservation corresponds to a command to start instances. If you launch two EC2 instance in one batch (for example, specifying the number of instances in the EC2 Console), this particular Reservation will have two EC2 instances. You can stop and started the EC2 instances to change their state, but this will not change their Reservation.

	public void listInstances()
	{
		System.out.println("\n\nLIST INSTANCE\n\n");
        	try 
		{
			// DescribeInstances
			DescribeInstancesResult result = client.describeInstances();

			// Traverse through the reservations
			List reservations = result.getReservations();
			for (Reservation reservation: reservations)
			{
				// Print out the reservation id
				String reservation_id = reservation.getReservationId();
				System.out.println("Reservation: " + reservation_id);
				// Traverse through the instances in a reservation
				List instances = reservation.getInstances();
				for (Instance instance: instances)
				{
					// Print out some information about the instance
					String id = instance.getInstanceId();
					String state = instance.getState().getName();
					System.out.println("\t" + id + "\t" + state);
				}
			}

	        } catch (Exception e) 
		{
			// Simple exception handling by printing out error message and stack trace
			System.out.println(e.getMessage());
			e.printStackTrace();
		}
	}

To terminate an EC2 instance, you will need to create a TerminateInstancesRequest object. The TerminateInstancesRequest object accepts a List of EC2 instance id through the setInstanceIds() method. Then you pass the TerminateInstancesRequest to the AmazonEC2Client’s terminateInstances() method, which returns a TerminateInstancesResult object. In the TerminateInstancesResult object, you have a List of InstanceStateChange, and each InstanceStateChange object contains information about the EC2 instance id, it’s previous state, and it’s current state.

	public void terminateInstance(String instanceId)
	{
		System.out.println("\n\nTERMINATE INSTANCE\n\n");
		try
		{
			// Construct the TerminateInstancesRequest
			TerminateInstancesRequest request = new TerminateInstancesRequest();
			ArrayList list = new ArrayList();
			list.add(instanceId);			// instance id
			request.setInstanceIds(list);

			// Pass the TerminateInstancesRequest to EC2
			TerminateInstancesResult result = client.terminateInstances(request);
			List  changes = result.getTerminatingInstances();
			for (InstanceStateChange change : changes)
			{
				String id = change.getInstanceId();
				String state_prev = change.getPreviousState().toString();
				String state_next = change.getCurrentState().toString();
				System.out.println("Instance " + id + " is changing from " + state_prev + " to " + state_next + ".");
			}
		} catch (Exception e)
		{
			// Simple exception handling by printing out error message and stack trace
			System.out.println(e.getMessage());
			e.printStackTrace();
		}
	}

Now you can make modifications to the demo code (in DemoEC2.java) with your preferred AWS region, AMI, instance type, along with other information. You can run the demo code using the following commands:

$ mvn compile
$ mvn package
$ java -cp target/demo-1.0-SNAPSHOT.jar net.qyjohn.aws.DemoEC2

This demo will launch one EC2 instance, list all the EC2 instances in the region, terminate the EC2 instance we just launched, and list all the EC2 instances in the region again. In order to demonstrate the state changes, we do a sleep of 10 seconds between each API call. After you have completed this exercise, you should intentionally introduce some errors in the code to observe the various exceptions thrown by the demo code. For example, you can point your AmazonEC2Client to use the us-east-1 region, but specify a subnet id or a security group id in another region.

[Logging Considerations]

When you encountered an error when making an API call using the AWS SDK for Java, the chance is that you have some mistake in the AWS resource you specified in the code, while the code itself is completely valid. In order to debug such issues, you will need to know what information is being sent to the AWS endpoint, and what information is returned from the AWs endpoint. In the dark age, debugging an application can only be achieved by looking into the Java object containing the request and result, then print out the information one by one using System.out.println(). Fortunately you don’t need to do this with the AWS SDK for Java, because the AWS SDK for Java is instrumented with Apache Commons Logging. All you need to do is grab a recent copy of log4j and set up the proper CLASSPATH, then ask your application to use the proper log4j configuration file.

In this demo, we have the JAR files for log4j version 2.3 in the third-party folder. we simply copy everything into our JRE’s lib/ext sub-folder. With this approach, you don’t need to worry about the CLASSPATH at all.

$ cd third-party
$ sudo cp *.jar /usr/lib/jvm/java-8-oracle/jre/lib/ext/

The log4j configuration file log4j2.xml, which is located at the top level folder of the demo code. You should refer to the log4j manual to understand how log4j works. To enable log4j for our demo application, you simply need to uncomment the Logger line in DemoEC2.java, as below:

public class DemoEC2 
{
	public AmazonEC2Client client;
	final static Logger logger = Logger.getLogger(DemoEC2.class);

After that, you will need to compile and package the demo code again. When running the Java application, you will need to pass the log4j configuration file to the java command:

$ mvn compile
$ mvn package
$ java -cp target/demo-1.0-SNAPSHOT.jar -Dlog4j.configurationFile=log4j2.xml net.qyjohn.aws.DemoEC2

As you can see, when you run the application, the request you send to the AWS endpoint, as well as the response from the AWS endpoint, are now display on your screen. With this information, it is a lot easier to debug your API calls to AWS endpoints.

By now we have completed the first chapter of this “Getting Started with AWS SDK for Java” tutorial. In the future I will publish more on this topic on a irregular base. Hopefully I will be able to cover the majority of AWS services that we use on a daily base. So, please stay tuned for my future updates.

婉清的歌

By , June 1, 2015 4:52 pm

云舒的歌

By , June 1, 2015 4:50 pm

悉尼的秋天

By , June 1, 2015 4:44 pm

IMG_0719

姊妹俩

IMG_0735

 

好漂亮的叶子哇!

 

 

Panorama Theme by Themocracy