S3 and Google Cloud?

Questions
1

Hey there! I’m trying to setup S3 backups but I plan to use Google Cloud Storage. Has anyone been able to set this up properly yet?

I know Google Cloud supports S3 auth but I can’t find anything related to it for the life of me. Maybe they’ve changed it? I don’t know

Thanks!

2

I know Google supports S3 buckets these days and that you can use the AWS SDK to access them at least to some extent, but don’t really know the details.

I got AMP to work with MinIO and learned a few things that might be helpful to you.

  • Use a DNS style (bucket.example.com) named bucket, not a path style (examble.com/bucket).
  • AMP uses the AWS SDK directly so if you can make the aws cli work with your bucket, then you’re probably good to go with AMP.

Following along to see the outcome here, good luck!

3

Howdy, were you ever able to figure out how to back up to google cloud?

4

Hi,

I’ve gotten pretty far with Cloud Storage and AMP. I’m currently running into the exact issue described here:

Stack Overflow: Expected hash not equal to calculated hash

According to the accepted answer, AMP’s uploading would need to add UseChunkEncoding = false to the PutObjectRequest.

5

AMP doesn’t use PutObjectRequest to upload files, it uses the TransferUtilityUploadRequest API. PutObjectRequest doesn’t work for very large transfers.

6

Thank you for your reply!

Though I am still running into the transfer failing with “Expected hash not equal to calculated hash”.

7

So far as I can tell this issue isn’t fixable from our end at all. The method outlined in the SO post won’t work for large files. So the only option I’m afraid is to use a different provider.

8

Stumbled upon this. I am using gaul s3proxy to then push to azure blob storage. It was working up until the 8th August and then it stopped and now I get the “Stack Overflow: Expected hash not equal to calculated hash” error. Not sure what changed, I’m guessing there was an AMP update.