AWS - Connecting from java applications
Reading material
In deployed environments
-
They will assign roles to the application.
-
Create a custom role for the application.
-
Create one or more custom policies for the application. If the application needs access to s3, sqs, elasticsearch, opensearch, and other services from aws, we will have to create the policies accordingly. e.g.
develop-application-name-s3-policy{ "Version":"2012-10-17", "Statement":[ { "Effect":"Allow", "Action": "s3:ListAllMyBuckets", "Resource":"*" }, { "Effect":"Allow", "Action":["s3:ListBucket","s3:GetBucketLocation"], "Resource":"arn:aws:s3:::amzn-s3-demo-bucket1" }, { "Effect":"Allow", "Action":[ "s3:PutObject", "s3:PutObjectAcl", "s3:GetObject", "s3:GetObjectAcl", "s3:DeleteObject" ], "Resource":"arn:aws:s3:::amzn-s3-demo-bucket1/*" } ] }or
{ "Version":"2012-10-17", "Statement":[ { "Action": "s3:*", "Effect":"Allow", "Resource":"*" } ] } -
Add all the policies necessary for the application to that role. Permission Policies:
develop-application-name-s3-policy(and whatever other policies the application needs). -
ProfileCredentialsProvidermight work great on your computer but won’t work anywhere that you’re getting AWS access through an IAM role (ex. Lambda, Docker, EC2 instance, etc). -
For applications deployed in AWS environments,
- the applications use some kind of credentials provider.
- On newer versions of the SDK, you’ll need to use following code
- This code will correctly pick up the active IAM role.
// To be used in deployed environments @Bean @Profile("!default") public AmazonS3 amazonS3() { return AmazonS3ClientBuilder.standard() .withCredentials(DefaultAWSCredentialsProviderChain.getInstance()) .build(); }
For local development
We cannot use IAM roles for application development in local computers. We have to IAM users.
- Create a custom user
- Create one or more custom policies for the user (similar to the way it is done for roles in the deployed environments). If the application needs access to s3, sqs, elasticsearch, opensearch, and other services from aws, we will have to create the policies accordingly.
How to set up AWS access credentials in your local?
https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/credentials.html
If you want to access AWS resources from your local machine using AWS SDK, you can’t do it by default.
You need to configure AWS access key id and AWS secret access key in your local.
You can do this using aws cli
Here is how to do it:
Run aws configure command in command prompt.
Open a command prompt and run aws configure command:
$ aws configure
AWS Access Key ID [None]: your-access-id
AWS Secret Access Key [None]: your-access-key
Default region name [None]: us-east-1
Default output format [None]:
Enter the AWS access key id and AWS secret access key when you are prompted for.
Also specify the default region you chose when you created an account in AWS.
That’s it!
Now two files get created in your home folder:
╭─explorer436@explorer436 in ~ on main [!?] on (us-east-1) took 1m9s
╰─λ cd ~/.aws
╭─explorer436@explorer436 in ~/.aws on main [!?] on (us-east-1)
╰─λ ls
.rw------- 29 explorer436 29 Jan 13:34 config
.rw------- 116 explorer436 29 Jan 13:34 credentials
You are all set now to access AWS resources from your local app!
Using AWSStaticCredentialsProvider
Avoid this if you can. Hard coding credentials in the application is a bad idea.
If you haven’t configured aws-cli in your local computer yet, and if you want to insert the aws access key and aws secret key in the application.properties file, the application can connect to AWS using this approach.
For this, the application.properties file has to look something like this:
bucket:
name: your-bucket-name
uploadExpirationTime: 3600000
downloadExpirationTime: 3600000
# https://cloud.spring.io/spring-cloud-static/spring-cloud-aws/2.2.0.M2/reference/html/
# https://docs.awspring.io/spring-cloud-aws/docs/current/reference/html/appendix.html
# AWS properties
cloud:
aws:
region:
static: us-east-1
stack:
auto: false
credentials:
accessKey: your-access-key
secretKey: your-secret-key
And then, using these in the configuration files in the application.
Sample config file:
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import com.amazonaws.auth.AWSStaticCredentialsProvider;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.regions.Regions;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;
@Configuration
public class AmazonS3ClientConfig {
@Value("${cloud.aws.credentials.accessKey}")
private String awsId;
@Value("${cloud.aws.credentials.secretKey}")
private String awsKey;
@Value("${cloud.aws.region.static}")
private String region;
@Bean
public AmazonS3 amazonS3Client() {
BasicAWSCredentials awsCredentials = new BasicAWSCredentials(awsId, awsKey);
AmazonS3 amazonS3Client = AmazonS3ClientBuilder.standard()
.withRegion(Regions.fromName(region))
.withCredentials(new AWSStaticCredentialsProvider(awsCredentials)).build();
return amazonS3Client;
}
}
Needless to say, this is not preferrable because it increases the risk of exposing your keys.
Reference:
ProfileCredentialsProvider
When the application uses ProfileCredentialsProvider, it looks in ~/.aws/credentials for a list of profiles. Using aws-cli (or manually), configure the user’s accessKey and secretKey for the default profile.
ProfileCredentialsProvider will read the two files ~/.aws/config and ~/.aws/credentials and use the parameters from these files to connect to aws.
The default credential profiles file - typically located at ~/.aws/credentials (location can vary per platform), is shared by many of the AWS SDKs and by the AWS CLI. The AWS SDK for Java uses the ProfileCredentialsProvider to load these credentials.
NOTE:
If you are using ProfileCredentialsProvider (see notes above), you don’t have to put credentials in property files. AWS SDK will read the credentials from environment files.
default profile
Configure aws-cli in your local computer and use the credentials from ~/.aws to connect to your AWS account from your applications.
If you have your credentials configured using aws-cli, you can use this config:
For this to work, the file /home/explorer436/.aws/credentials should have
[default]
aws_access_key_id = <my_aws_access_key_id>
aws_secret_access_key = <my_access_key>
The config class in java application should look like this:
import com.amazonaws.auth.profile.ProfileCredentialsProvider;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
@Configuration
public class AmazonS3ClientConfig {
@Bean
public AmazonS3 amazonS3Client() {
AmazonS3 amazonS3Client = AmazonS3ClientBuilder
.standard()
.withCredentials(new ProfileCredentialsProvider("default"))
.build();
return amazonS3Client;
}
}
saml profile
When working in a corporate environment, we may have to use a combination of saml profile and proxy credentials as opposed to default profile. The details vary by clients.
import com.amazonaws.auth.profile.ProfileCredentialsProvider;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
@Configuration
public class AmazonS3ClientConfig {
@Bean
@Profile("default")
public AmazonS3 amazonS3() {
amazonS3 = AmazonS3ClientBuilder
.standard()
.withClientCredentials(new ProfileCredentialsProvider("saml"))
.withRegion(Regions.fromName(awsRegion))
.withClientConfiguration(
PredefinedClientConfigurations.defaultConfig()
.withProxyHost(proxyHost)
.withProxyPort(Integer.parseInt(proxyPort))
.withProxyUsername(proxyUserName)
.withProxyPassword(proxyPassword)
)
.build();
return amazonS3Client;
}
}
Using EndpointConfiguration
This is an example using Localstack: