Solution Guide: Securing AWS Zero Trust Access to S3 Buckets with Python, VPC, and Ziti SDK
NetFoundry AWS Zero Trust
NetFoundry AWS Zero Trust offers private, zero trust networking to S3 buckets for apps, devices, and users
In this NetFoundry solution guide, learn how to securely access your S3 buckets using VPC endpoints, the Boto3 S3 client, and NetFoundry’s Ziti SDK. This guide outlines the process to integrate NetFoundry’s secure cloud network for high performance and security in your AWS environment. You’ll learn how AWS Zero Trust principles enhance security by embedding zero trust connectivity, ensuring secure access and protection for modern cloud environments
The solution described in this guide uses NetFoundry Cloud which makes it easy to instantly spin up highly secure, performant, edge, app or device -to-cloud networks for workloads in AWS. Our secure private overlays on the internet offer private, zero trust networking to S3 bucket and objects for apps, devices and users.
With NetFoundry, you can extend secure zero trust connections to S3 buckets and objects that are not public, following least privilege access and micro segmentation principles. Smart fabric from NetFoundry provides you with optimal latency routes for your apps. This ensures you can build highly secure and performant connectivity in minutes using cloud-native tools without the burden of a direct connect solution.
This guide explores how to leverage NetFoundry to establish secure access for your private Amazon S3 buckets from a python log pusher program running a Boto3 Client for S3. We’ll establish the private overlay using a NetFoundry edge router in AWS and a NetFoundry Python SDK imported in the Py log pusher program. You can follow the approach for any Py based app or explore our SDKs for other programming languages. We would be exploring how to access the bucket and objects from a laptop that is running a NetFoundry Ziti Desktop Edge software client.
Secure Cloud Access
Discover how to implement private, zero trust networking for Amazon S3 buckets using NetFoundry’s AWS solution.
This guide provides step-by-step instructions for integrating VPC endpoints and the Boto3 S3 client, ensuring secure access and protection for cloud environments.
Getting Started Guide
Kick off your NetFoundry journey with a free trial account! Ensure you have a network set up with at least one public router and follow our guides to meet firewall policy requirements for secure outbound access.
AWS Zero Trust: Getting Started
What you need to get started:
- A NetFoundry Cloud account
- Go through the steps to create a free trial account if you don’t have one.
- A network in your account with at least one public router. The articles will guide you through this process.
- How to create a network
- Provision a public router ( NetFoundry hosted)
- Firewall policy requirements to provide outbound only access to the NetFoundry network
Solution Architecture:
Setting up S3 bucket, VPC interface endpoint and policies in AWS
A. S3 Bucket:
You can follow the steps outlined in the following document to create a S3 bucket.
https://docs.aws.amazon.com/AmazonS3/latest/userguide/creating-bucket.html
Create the bucket that is not public.
B. Interface VPC endpoint:
- Create a new VPC or use an existing VPC to provision your VPC endpoint.
- Create an interface VPC endpoint to controls access to S3 bucket. The S3 bucket would be reached within the VPC via the interface VPC endpoint.
- Refer the following document for details on using interface VPC endpoint to access a S3 bucket: https://docs.aws.amazon.com/vpc/latest/privatelink/create-interface-endpoint.html
Interface VPC endpoint are represented by one or more elastic network interfaces (ENIs) that are assigned private IP addresses from subnets in your VPC which will be used to configure NetFoundry service.
You can attach an endpoint policy to your VPC endpoint that controls access to Amazon S3. The policy in the snapshot below provides the interface VPCE access to all resources within the VPC.
C. S3 Bucket Policy:
Once the VPCe and S3 bucket are provisioned, the next step is to provision the S3 bucket policy. With Amazon S3 bucket policies, you can secure access to objects in your buckets, so that only services or users with the appropriate permissions can access them. For this demo, we’re using a bucket policy to restrict access by a VPC endpoint . Below is a bucket policy that allows access to the specific VPCe ” vpce-0db2ff4e77e2622ba” to perform actions on the S3 bucket “sdktestingwithboto3” and its objects.
Setting Up S3 Access
Securely set up your S3 bucket by creating it as private, configuring an interface VPC endpoint, and applying a bucket policy that restricts access to authorized services only.
Deploying NetFoundry Edge Router
Set up the NetFoundry edge router as your WAN gateway within the same VPC as your interface VPC endpoint, ensuring secure access to your S3 bucket.
Spin up a NetFoundry edge router in AWS
The NetFoundry edge router is the WAN gateway in the VPC that helps you to reach the S3 bucket via the VPC endpoint over a private and secure zero trust overlay. The ER is deployed on a EC2 instance. You can also deploy the ER in a container.
Follow the instructions to spin up the NetFoundry edge router in AWS from the AWS marketplace. The router has to be spun up in the same VPC as your interface VPC endpoint or have the reachability to the VPC endpoint if it is spun up in a different VPC.
The router should show registered and online once provisioned successfully.
Create your identity, service and service policy
- You can access your S3 bucket via a NetFoundry endpoint software on your laptop or via ziti python SDK embedded into your application that requires access to S3 bucket.
- Create your identity each to access the S3 bucket from the NetFoundry Ziti Py SDK and the Ziti desktop edge
- Create your service for accessing S3 bucket via interface VPC endpoint.The following URL is used for private access to S3 buckets associated with the interface VPC endpoint from the boto3 client for S3 imported in the python program.
https://sdktestingwithboto3.vpce-0db2ff4e77e2622ba-uf7ato7f.s3.ap-southeast-1.vpce.amazonaws.com
- From the laptop running ziti desktop edge, the following URL is used to access the object netfoundry.jpg
https://sdktestingwithboto3.s3.ap-southeast-1.amazonaws.com/netfoundry.jpg
The service configuration is configured with the “wildcard” domain name using the DNS name of the interface VPC endpoint and the private IP address attached to elastic network interface (ENI) .
The identity is that of the customer edge router that was provisioned in AWS from the marketplace.
Port 443 has been selected for private access within the VPC via the interface VPC endpoint.
- Create your service policy to allow your identity (or identities) to access the S3 Access service.
Create a service policy to allow the identities for devices or the router identity deployed in your factory or site to access the S3 bucket over the highly secure NetFoundry cloud network.
The service policy that allows identities to access the S3 bucket has been created as shown below:
Accessing S3 with NetFoundry
Access your S3 bucket securely using the NetFoundry Ziti SDK or the Ziti Desktop Edge.
Secure S3 Access
Easily access AWS S3 private buckets using Python with the Ziti SDK. Download the demo program to generate log files and upload them securely over the NetFoundry Cloud network. Start by enrolling your identity and running the provided commands.
Accessing S3 bucket and objects using a Boto3 client + Ziti Python SDK over the NetFoundry Cloud
Download demo python program that imports AWS’s boto3 and ziti SDK to access the private buckets via the Ziti. The Py program generates logfiles and uploads them to the S3 bucket, creating a folder and uploading the log files to the folder over the NetFoundry Cloud network.
https://github.com/openziti-test-kitchen/boto-demo/tree/main
Use the following commands to run the Py program. You can run export ZITI_LOG=4 to view the logs:
python boto-demo-main/s3z/s3z.py \
–ziti-identity-file “/d/S3/identityname.json” \
–bucket-name “sdktestingwithboto3” \
–bucket-endpoint “https://bucket.vpce-0db2ff4e77e2622ba-uf7ato7f.s3.ap-southeast-1.vpce.amazonaws.com” \
–push-log-dir “logs” \
–object-prefix “foldername”
The identityname.json is the json file of the identity that is registered with the NetFoundry network.
How to enroll the jwt obtained from the console and generate the json:
python -m openziti enroll –jwt identityname.jwt –identity identityname.json
https://github.com/openziti/ziti-sdk-py/blob/main/sample/README.md#get-and-enroll-an-identity
The bucket name is the name of your S3 bucket.
The bucket-endpoint is the DNS of the S3 VPC endpoint prefixed with your bucket name.
The object-prefix is the name of the folder you want to create in S3.
Once you execute the Py program, you will find the output like the one below:
The folder is created in your S3 bucket:
And the log files are uploaded in the folder:
Accessing S3 bucket and objects using a Ziti desktop edge over the NetFoundry Cloud
Install the desktop edge based on the OS of your device: https://netfoundry.io/downloads/
You’ll find the installation instructions for the endpoints in the respective sections.
Register your identity to the network from your endpoint software.
With the required service policy in place, you should see the S3 service listed on your endpoint.
Access the S3 object from your device over the NetFoundry network:
Unlock S3 Connectivity
Install the desktop edge for your OS and gain seamless access to S3 objects via the NetFoundry network. Register your identity and follow the instructions to begin.
AWS Zero Trust Security
Adopt AWS Zero Trust to enhance your cloud security, prevent unauthorized access, and protect your applications and data in a complex digital landscape.
Conclusion: The Power of AWS Zero Trust
Incorporating AWS Zero Trust principles into your cloud infrastructure ensures that your systems are protected by advanced, modern security measures. By embedding zero trust connectivity, organizations can prevent unauthorized access and reduce the risk of security breaches, all while maintaining seamless operations. AWS Zero Trust empowers businesses to secure their applications, data, and users in an increasingly complex digital landscape.
