diff --git a/README.md b/README.md index 6073e83..d018907 100644 --- a/README.md +++ b/README.md @@ -45,6 +45,32 @@ configure beyond the bucket. Missing or forbidden keys return an honest `404` / `403` with an XML error body (no v3-style empty-`200`). +## Deploy on AWS + +[`deploy/aws-ecs/`](./deploy/aws-ecs/) is a reference CloudFormation stack that +runs this image on **AWS Fargate** behind an Application Load Balancer, with a +task role scoped to `s3:GetObject` on your bucket, `/health` health checks, and +CPU autoscaling. Bring your own VPC and subnets; HTTPS and a Route 53 alias are +optional: + +```bash +aws cloudformation deploy \ + --stack-name s3proxy \ + --template-file deploy/aws-ecs/s3proxy-fargate.yaml \ + --capabilities CAPABILITY_IAM \ + --parameter-overrides \ + BucketName=my-bucket \ + VpcId=vpc-0123456789abcdef0 \ + SubnetIds=subnet-aaaa,subnet-bbbb \ + ContainerImage=forkzero/s3proxy:4.2 +``` + +See [`deploy/aws-ecs/README.md`](./deploy/aws-ecs/README.md) for parameters, +HTTPS/DNS setup, and how to try it against the public `s3proxy-public` demo +bucket. For other targets, the container is a standard image — run it under +Compose, Kubernetes, or any scheduler that can pass the `BUCKET` env var and an +IAM role/credentials. + ## Local development To run against a bucket that requires credentials, mint a short-lived session diff --git a/deploy/aws-ecs/README.md b/deploy/aws-ecs/README.md new file mode 100644 index 0000000..2701ced --- /dev/null +++ b/deploy/aws-ecs/README.md @@ -0,0 +1,117 @@ +# Deploy s3proxy on AWS Fargate + +A reference CloudFormation stack that runs the [`forkzero/s3proxy`](https://hub.docker.com/r/forkzero/s3proxy) +container on AWS Fargate behind an Application Load Balancer, streaming objects +from an S3 bucket you name. + +[`s3proxy-fargate.yaml`](./s3proxy-fargate.yaml) provisions: an ALB (+ security +group), a listener (HTTPS/443 if you supply an ACM certificate, otherwise +HTTP/80), a target group health-checked on `/health`, an ECS cluster, a Fargate +service + task definition, a **task role scoped to `s3:GetObject` on your bucket +only**, a CloudWatch log group, and CPU target-tracking autoscaling. HTTPS and a +Route 53 alias are optional. + +Nothing in the template is account-specific — you pass your VPC, subnets, and +bucket as parameters. + +## Prerequisites + +- A VPC and **two or more subnets in different AZs**. Public subnets are + simplest (tasks get a public IP to reach S3); private subnets work if they + have a NAT gateway. +- The AWS CLI, authenticated to the target account. +- The bucket already exists. + +## Deploy + +```bash +aws cloudformation deploy \ + --stack-name s3proxy \ + --template-file s3proxy-fargate.yaml \ + --capabilities CAPABILITY_IAM \ + --parameter-overrides \ + BucketName=my-bucket \ + VpcId=vpc-0123456789abcdef0 \ + SubnetIds=subnet-aaaa,subnet-bbbb \ + ContainerImage=forkzero/s3proxy:4.2 +``` + +`CAPABILITY_IAM` is required because the stack creates the task and execution +roles. When it finishes, read the URL from the outputs: + +```bash +aws cloudformation describe-stacks --stack-name s3proxy \ + --query "Stacks[0].Outputs[?OutputKey=='Endpoint'].OutputValue" --output text +curl "$(aws cloudformation describe-stacks --stack-name s3proxy \ + --query "Stacks[0].Outputs[?OutputKey=='Endpoint'].OutputValue" --output text)/index.html" +``` + +### Try it against the public demo bucket + +`s3proxy-public` is a world-readable demo bucket. Point the stack at it to see a +working deployment without creating data of your own: + +```bash +aws cloudformation deploy \ + --stack-name s3proxy-demo \ + --template-file s3proxy-fargate.yaml \ + --capabilities CAPABILITY_IAM \ + --parameter-overrides \ + BucketName=s3proxy-public \ + VpcId=vpc-0123456789abcdef0 \ + SubnetIds=subnet-aaaa,subnet-bbbb +``` + +## Enable HTTPS and a custom domain (optional) + +Supply an ACM certificate ARN (in the **same region** as the stack) to switch +the listener to HTTPS/443, and a domain + hosted zone to add a Route 53 alias: + +```bash + --parameter-overrides \ + BucketName=my-bucket \ + VpcId=vpc-0123456789abcdef0 \ + SubnetIds=subnet-aaaa,subnet-bbbb \ + CertificateArn=arn:aws:acm:us-east-1:123456789012:certificate/uuid \ + DomainName=s3proxy.example.com \ + HostedZoneName=example.com +``` + +## Parameters + +| Parameter | Default | Notes | +| ---------------- | ----------------------- | ------------------------------------------------------------ | +| `BucketName` | *(required)* | Bucket to serve; the task role gets `s3:GetObject` on it. | +| `VpcId` | *(required)* | VPC to deploy into. | +| `SubnetIds` | *(required)* | ≥2 subnets in different AZs (comma-separated). | +| `ContainerImage` | `forkzero/s3proxy:latest` | **Pin a version in production**, e.g. `forkzero/s3proxy:4.2`. | +| `ContainerPort` | `8080` | Passed to the image as `PORT`. | +| `DesiredCount` | `2` | Tasks to run. | +| `MinCapacity` / `MaxCapacity` | `1` / `4` | Autoscaling bounds (CPU target 70%). | +| `Cpu` / `Memory` | `1024` / `2048` | Fargate task size (must be a valid CPU/memory pair). | +| `CertificateArn` | `""` | ACM cert ARN → HTTPS/443. Blank → HTTP/80. | +| `DomainName` / `HostedZoneName` | `""` / `""` | Route 53 alias to the ALB. Blank → no DNS record. | + +## Update the image / roll out + +Redeploy with a new `ContainerImage`, or force a fresh pull of a moving tag: + +```bash +aws ecs update-service --cluster s3proxy --service s3proxy --force-new-deployment +``` + +## Tear down + +```bash +aws cloudformation delete-stack --stack-name s3proxy +``` + +## Notes + +- **This is a starting point, not a turnkey production stack.** Review the + security group (open to `0.0.0.0/0`), TLS policy, log retention, and scaling + bounds for your needs. +- The task role grants read-only `s3:GetObject` on the one bucket — the proxy + needs nothing more. +- Other platforms (plain `docker run`, Compose, Kubernetes) are covered in the + [main README](../../README.md). diff --git a/deploy/aws-ecs/s3proxy-fargate.yaml b/deploy/aws-ecs/s3proxy-fargate.yaml new file mode 100644 index 0000000..5435056 --- /dev/null +++ b/deploy/aws-ecs/s3proxy-fargate.yaml @@ -0,0 +1,288 @@ +AWSTemplateFormatVersion: 2010-09-09 +Description: >- + Reference deployment of the forkzero/s3proxy container on AWS Fargate behind + an Application Load Balancer. Streams objects from an S3 bucket over HTTP(S). + Bring your own VPC and subnets; HTTPS + DNS are optional (provide an ACM + certificate and hosted zone to enable them). Nothing here is account-specific. + +Parameters: + BucketName: + Type: String + Description: S3 bucket the proxy serves (the task role is granted s3:GetObject on it). + ContainerImage: + Type: String + Default: forkzero/s3proxy:latest + Description: Image to run. Pin a version in production, e.g. forkzero/s3proxy:4.2. + VpcId: + Type: AWS::EC2::VPC::Id + Description: VPC to deploy into. + SubnetIds: + Type: List + Description: >- + Two or more subnets in different AZs for the ALB and tasks. Use public + subnets (tasks get a public IP to reach S3), or private subnets with a + NAT gateway. + ContainerPort: + Type: Number + Default: 8080 + Description: Port the container listens on (passed to the image as PORT). + DesiredCount: + Type: Number + Default: 2 + Description: Number of tasks to run. + MinCapacity: + Type: Number + Default: 1 + Description: Minimum tasks for autoscaling. + MaxCapacity: + Type: Number + Default: 4 + Description: Maximum tasks for autoscaling. + Cpu: + Type: String + Default: "1024" + Description: Fargate task CPU units (256, 512, 1024, 2048, 4096). + Memory: + Type: String + Default: "2048" + Description: Fargate task memory (MiB), valid for the chosen CPU. + CertificateArn: + Type: String + Default: "" + Description: >- + Optional ACM certificate ARN. If set, the ALB listens on HTTPS/443; + otherwise it listens on HTTP/80. + DomainName: + Type: String + Default: "" + Description: >- + Optional fully-qualified name to alias to the ALB, e.g. s3proxy.example.com. + Requires HostedZoneName. Leave blank to skip DNS. + HostedZoneName: + Type: String + Default: "" + Description: >- + Optional Route 53 hosted zone the DomainName lives in, e.g. example.com + (no trailing dot). + +Conditions: + UseHttps: !Not [!Equals [!Ref CertificateArn, ""]] + CreateDns: !Not [!Equals [!Ref DomainName, ""]] + +Resources: + LoadBalancerSecurityGroup: + Type: AWS::EC2::SecurityGroup + Properties: + GroupDescription: !Sub "${AWS::StackName} ALB ingress" + VpcId: !Ref VpcId + SecurityGroupIngress: + - IpProtocol: tcp + FromPort: !If [UseHttps, 443, 80] + ToPort: !If [UseHttps, 443, 80] + CidrIp: 0.0.0.0/0 + + ContainerSecurityGroup: + Type: AWS::EC2::SecurityGroup + Properties: + GroupDescription: !Sub "${AWS::StackName} task ingress from ALB" + VpcId: !Ref VpcId + SecurityGroupIngress: + - IpProtocol: tcp + FromPort: !Ref ContainerPort + ToPort: !Ref ContainerPort + SourceSecurityGroupId: !Ref LoadBalancerSecurityGroup + + LoadBalancer: + Type: AWS::ElasticLoadBalancingV2::LoadBalancer + Properties: + Scheme: internet-facing + Type: application + SecurityGroups: + - !Ref LoadBalancerSecurityGroup + Subnets: !Ref SubnetIds + LoadBalancerAttributes: + - Key: idle_timeout.timeout_seconds + Value: "60" + + TargetGroup: + Type: AWS::ElasticLoadBalancingV2::TargetGroup + Properties: + VpcId: !Ref VpcId + Port: !Ref ContainerPort + Protocol: HTTP + TargetType: ip + HealthCheckPath: /health + HealthCheckIntervalSeconds: 30 + HealthCheckTimeoutSeconds: 5 + HealthyThresholdCount: 2 + UnhealthyThresholdCount: 2 + TargetGroupAttributes: + - Key: deregistration_delay.timeout_seconds + Value: "60" + + # One listener, shaped by whether a certificate was supplied: HTTPS/443 with + # the cert, else HTTP/80. A single (unconditional) resource so the Service can + # DependsOn it — the target group must be attached to a listener before the + # service registers targets. + Listener: + Type: AWS::ElasticLoadBalancingV2::Listener + Properties: + LoadBalancerArn: !Ref LoadBalancer + Port: !If [UseHttps, 443, 80] + Protocol: !If [UseHttps, HTTPS, HTTP] + Certificates: !If + - UseHttps + - - CertificateArn: !Ref CertificateArn + - !Ref AWS::NoValue + DefaultActions: + - Type: forward + TargetGroupArn: !Ref TargetGroup + + LogGroup: + Type: AWS::Logs::LogGroup + Properties: + LogGroupName: !Sub "/ecs/${AWS::StackName}" + RetentionInDays: 30 + + # Grants the running container read access to the bucket it serves. + TaskRole: + Type: AWS::IAM::Role + Properties: + AssumeRolePolicyDocument: + Statement: + - Effect: Allow + Principal: + Service: ecs-tasks.amazonaws.com + Action: sts:AssumeRole + Policies: + - PolicyName: s3-read + PolicyDocument: + Version: "2012-10-17" + Statement: + - Effect: Allow + Action: s3:GetObject + Resource: !Sub "arn:${AWS::Partition}:s3:::${BucketName}/*" + + # Lets ECS pull the image and write logs (managed AWS policy). + ExecutionRole: + Type: AWS::IAM::Role + Properties: + AssumeRolePolicyDocument: + Statement: + - Effect: Allow + Principal: + Service: ecs-tasks.amazonaws.com + Action: sts:AssumeRole + ManagedPolicyArns: + - !Sub "arn:${AWS::Partition}:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy" + + Cluster: + Type: AWS::ECS::Cluster + Properties: + ClusterName: !Ref AWS::StackName + CapacityProviders: + - FARGATE + + TaskDefinition: + Type: AWS::ECS::TaskDefinition + Properties: + Family: !Ref AWS::StackName + Cpu: !Ref Cpu + Memory: !Ref Memory + NetworkMode: awsvpc + RequiresCompatibilities: + - FARGATE + ExecutionRoleArn: !GetAtt ExecutionRole.Arn + TaskRoleArn: !GetAtt TaskRole.Arn + ContainerDefinitions: + - Name: s3proxy + Image: !Ref ContainerImage + Essential: true + PortMappings: + - ContainerPort: !Ref ContainerPort + Protocol: tcp + Environment: + - Name: BUCKET + Value: !Ref BucketName + - Name: PORT + Value: !Ref ContainerPort + - Name: AWS_NODEJS_CONNECTION_REUSE_ENABLED + Value: "1" + LogConfiguration: + LogDriver: awslogs + Options: + awslogs-region: !Ref AWS::Region + awslogs-group: !Ref LogGroup + awslogs-stream-prefix: ecs + + Service: + Type: AWS::ECS::Service + DependsOn: Listener + Properties: + Cluster: !Ref Cluster + ServiceName: s3proxy + TaskDefinition: !Ref TaskDefinition + LaunchType: FARGATE + DesiredCount: !Ref DesiredCount + HealthCheckGracePeriodSeconds: 60 + LoadBalancers: + - ContainerName: s3proxy + ContainerPort: !Ref ContainerPort + TargetGroupArn: !Ref TargetGroup + NetworkConfiguration: + AwsvpcConfiguration: + AssignPublicIp: ENABLED + SecurityGroups: + - !Ref ContainerSecurityGroup + Subnets: !Ref SubnetIds + + AutoScalingTarget: + Type: AWS::ApplicationAutoScaling::ScalableTarget + Properties: + MinCapacity: !Ref MinCapacity + MaxCapacity: !Ref MaxCapacity + ResourceId: !Sub "service/${Cluster}/${Service.Name}" + ScalableDimension: ecs:service:DesiredCount + ServiceNamespace: ecs + # AWS-managed service-linked role for ECS autoscaling; no IAM role to define. + RoleARN: !Sub "arn:${AWS::Partition}:iam::${AWS::AccountId}:role/aws-service-role/ecs.application-autoscaling.amazonaws.com/AWSServiceRoleForApplicationAutoScaling_ECSService" + + AutoScalingPolicy: + Type: AWS::ApplicationAutoScaling::ScalingPolicy + Properties: + PolicyName: cpu-target-tracking + PolicyType: TargetTrackingScaling + ScalingTargetId: !Ref AutoScalingTarget + TargetTrackingScalingPolicyConfiguration: + PredefinedMetricSpecification: + PredefinedMetricType: ECSServiceAverageCPUUtilization + TargetValue: 70 + ScaleInCooldown: 60 + ScaleOutCooldown: 60 + + DnsRecord: + Type: AWS::Route53::RecordSet + Condition: CreateDns + Properties: + HostedZoneName: !Sub "${HostedZoneName}." + Name: !Sub "${DomainName}." + Type: A + AliasTarget: + DNSName: !GetAtt LoadBalancer.DNSName + HostedZoneId: !GetAtt LoadBalancer.CanonicalHostedZoneID + +Outputs: + LoadBalancerDns: + Description: Public DNS name of the load balancer. + Value: !GetAtt LoadBalancer.DNSName + Endpoint: + Description: URL to reach the proxy. + Value: !If + - CreateDns + - !Sub + - "${Scheme}://${DomainName}" + - Scheme: !If [UseHttps, https, http] + - !Sub + - "${Scheme}://${Dns}" + - Scheme: !If [UseHttps, https, http] + Dns: !GetAtt LoadBalancer.DNSName