What You Need Before You Start :
- AWS CLI installed on your computer
- Your AWS Access Key and Secret Key (shared with you separately)
- The S3 folder path where your data dump is stored
1. Install AWS CLI (If You Don’t Have It Yet)
AWS CLI lets your computer talk to the AWS servers.
You can download and install it from here:
https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html
Follow the instructions based on your operating system. After installation, continue to the next step.
2. Set Up AWS CLI on Your System
You only need to do this once.
- Open your Terminal (Mac) or Command Prompt / PowerShell (Windows).
-
Type this and press Enter:
aws configure
-
Enter the details you’ve received:
- AWS Access Key ID
- AWS Secret Access Key
- Default region:
us-east-1 - Default output format:
json
Your computer is now authorized to access the S3 bucket.
3. Create a Folder to Save the Data Dump
Now choose where you want the data to be stored.
In your terminal, type:
cd Downloads
mkdir datadump
You can name the folder anything you like.
4. Download the Data Dump from S3
This is the main step.
Run this command, replacing <your-s3-folder-path> with the actual path you received:
aws s3 sync <your-s3-folder-path> datadump
For example (format only):
aws s3 sync s3://bucket-name/folder-path/ datadump
- sync means copy
- datadump is your local folder
Hit Enter, and the files will start downloading to your folder.
Repeat the same command if you have multiple S3 paths.
5. (Optional) Make the Data Easier to Read
If your data dump contains JSON files, you can format them to make them readable.
Here’s a simple way using Sublime Text:
Install Sublime Text
Download from: https://www.sublimetext.com/
Add Pretty JSON Plugin
- Open Sublime
- Press Cmd + Shift + P (Mac) or Ctrl + Shift + P (Windows)
- Search: Package Control: Install Package
- Install it
- Open the command palette again
- Search for Pretty JSON and install it
Format Your JSON
- Open the file in Sublime
- Press Cmd/Ctrl + A (Select All)
- Open command palette
- Search: Pretty JSON: Format JSON Lines
Your data will now look neat and easy to review.