Access to S3
Completion requirements
This notebook provides a comprehensive tutorial on using the boto3 library in Python to access and download data from S3-compatible object storage.
Steps covered:
- Installation & Imports: Setting up
boto3and configuring it forUNSIGNEDaccess, which allows you to retrieve public data without needing AWS credentials. - Resource vs. Client: Demonstrating the two ways to interact with S3:
- Resource: A higher-level, object-oriented interface (e.g., using
s3r.Bucket). - Client: A lower-level interface that maps closely to the actual service API (e.g., using
s3c.list_objects).
- Resource: A higher-level, object-oriented interface (e.g., using
- Data Exploration: How to list buckets, iterate through objects, and inspect metadata like file sizes and keys.
- File Operations: Practical examples of downloading files (like
README.txtand3u2023.json) directly to the local environment and reading their contents.
The examples specifically use the GFZ Potsdam S3 endpoint to explore DAS datasets.
Click on Access to S3 to open the resource.