Send Large Files with Amazon S3 and Groovy

Sometimes you need to send a very large files to people. You could use your email or file sharing service of choice, but they have limits.  For example:

Google Mail - 25 MB limit
Google Drive - 100 GB limit
Dropbox - 2-16 GB limit

Even if your file is within the limit, you don't necessarily want to use those services each time, because it will add up, and eventually you'll run out of space in your account.

You could use services that specialize in sending large files. For example: Hightail, WeTransfer, DropSend, etc. But, you may not trust them with your files or just don't want to bother with creating additional accounts.

I came up with another solution that works well for developers like me that are familiar with using Amazon S3.  With a simple script (I wrote mine in Groovy), I can upload any file from my computer to Amazon S3 and then generate a pre-signed URL that gives access to the file for a certain period of time.  This URL can be IM'd or emailed to anyone so that they can download the file before the URL expires.

Here are the steps to use this script:

1. Make sure you have an AWS account and a bucket setup in S3.
2. Download the script from GitHub: UploadToS3AndGenerateSignedUrl.groovy
3. Modify the global variables:
• pathToFile
• bucketName
• expiresInHours
• myAccessKeyID,
• mySecretKey
4. Run the script:
• groovy UploadToS3AndGenerateSignedUrl.groovy
5. Copy the URL that gets printed to stdout. It will look similar to this:
• https://my-bucket-name.s3.amazonaws.com/myLargeFile.zip?AWSAccessKeyId=12E3AZEDFJQ6CMZW1HG5&Expires=1421887816&Signature=GFQ0LeR%2FtgWWGtXu3cJXspFtKNE%3D
6. Send the URL to someone else in an email. They can click on it to download the file.
7. Optionally, use the Amazon S3 Console to remove the file from S3 once the recipient(s) have downloaded the file so that you don't get charged to have it sitting there forever.