FineUploader 5.9 S3 multipart version 4 Signature Mismatch - bucket Name

[nsmaminebelhassine]

Type of issue

Bug

Uploader type

S3

Bug details

Fine Uploader version

5.9

Browsers where the bug is reproducible

Chrome Version 50.0.2661.102 (64-bit)

Operating systems where the bug is reproducible

OSX 10.10.3

Exact steps required to reproduce the issue

1. Select a big file to upload

All of your Fine Uploader initialization JavaScript code

    signature: {
        endpoint: "aws/signv4",
        version: 4
    },

    objectProperties: {
        key: function (fileId) {
            var dir = $("#dir").val();
            var filename = $('#fineuploader-s3').fineUploader('getName', fileId);
            var uuid = $('#fineuploader-s3').fineUploader('getUuid', fileId);
            var ext = filename.substr(filename.lastIndexOf('.') + 1);
            return  dir + uuid + '.' + ext;
        },
        region: $("#region").val()
    },

Detailed explanation of the problem

I get the following error from AWS :

SignatureDoesNotMatchThe request signature we calculated does not match the signature you provided. Check your key and signing method.

After some investigation i found that signature does not match because :

1/ In the first request : the sign request ( fineuploader -> backend sign url ) the headers value is the following :

"AWS4-HMAC-SHA256↵20160607T123918Z↵20160607/eu-central-1/s3/aws4_request↵POST↵/0PbAIa/UqRA9a/7b8b152a-13da-4a1e-9ef6-1123289c517a.pdf↵uploads=↵host:s3-eu-central-1.amazonaws.com ........."

Note that after the POST the bucket name is not included

2/ In the third request : the upload request ( fineuploader -> POST to aws s3 ) i get the 403 error SignatureDoesNotMatch and in the details i can read that my canonical request is :

POST
/my-bucket-name/0PbAIa/UqRA9a/7b8b152a-13da-4a1e-9ef6-1123289c517a.pdf
uploads=
host:s3-eu-central-1.amazonaws.com
.......

=> The filename between the sign request and the upload request is different : in the upload request it contains the bucket name, but in the sign request it does not. So the signature is different between the two requests.

So how the key is passed by fineuploader to the sign endpoint ? is there a chance that in some conditions the bucket name is not passed ? what are these conditions ?
I can correct this by a workaround in the server side sign code ( derived from fineuploader php aws sign code and modified to fit on our project ) to include the bucket name in the file key but it does not seems to be the clean way to do it.

[nsmaminebelhassine]

If fixed that by changing the getCanonicalUri method ( applicable only for v4 and chunked uploads if I understand the code ) :
if added the bucket name before the file uri
now the request goes clean to the sign process

          getCanonicalUri: function(endOfUri) {
                var path = endOfUri,
                    queryParamIdx = endOfUri.indexOf("?");

                if (queryParamIdx > 0) {
                    path = endOfUri.substr(0, queryParamIdx);
                }
                return escape("/"+ "my-bucket-name" +"/"+ decodeURIComponent(path));
            },

Now i'm searching how to make it dynamically read the actual bucket name.
But the question is : Is this coherent and correct ? As a workaround it works perfectly but i don't like to ship with changed code on 3rd party librairies.

I encountred another similar problem with the host and corrected it with just providing the "host" in the objectProperties.

[rnicholus]

There is a bug somewhere in the code (either yours or Fine Uploader's), but the issue is very specific to your setup. In other words, v4 signatures are certainly not broken for everyone. I suspect the fact that you have a key function is important to locating the source of this bug. Perhaps if you remove it, everything works as expected. That may be the path to follow in order to determine exactly what the bug is here, or if one exists.

[nsmaminebelhassine]

I removed the key function and changed it to the "uuid" value

        objectProperties: {
            key: "uuid",
            region: $("#region").val(),
            host: "s3-eu-central-1.amazonaws.com"
        },

Same problem :
Fineuploader -> backend sign url :

{"headers":"AWS4-HMAC-SHA256\n20160607T135713Z\n20160607/eu-central-1/s3/aws4_request\nPOST\n/c4d88dd0-8102-4a53-b161-5b203f263d62.pdf\nuploads...."

Fineuploader -> AWS S3
AWS Canonical Request :

<CanonicalRequest>POST
/my-bucket-name/c4d88dd0-8102-4a53-b161-5b203f263d62.pdf
uploads=
....
</CanonicalRequest>

Same if i change the key value to "filename"

I had the same reasoning at the beginning but seems to be unrelated.

And yes v4 signatures are not broken for everyone : i didn't found the same problem elsewhere.

[rnicholus]

Then the next step is to determine exactly what is causing the issue in your situation, that is, that unique factor in your environment, code, or setup. Once we know that, it will be possible to address the issue.

[rnicholus]

Closing this due to lack of activity for an extended period of time. I'd be happy to consider reopening if more information is provided.

[genestd]

I'm having this EXACT problem, and can't figure out where to look for the solution. Is it possible for you to point me to code where the S3 object key to be uploaded is defined? FineUploader is using a different value for the request to the signing endpoint vs. the POST request to the S3 endpoint - specifically it is adding the bucket name to the object key when POSTing to S3.

[chris-denning]

Hi,

I'm hitting the same problem, and have done some debugging. Hopefully this will point you in the right direction for a fix.

The problem seems to affect the "bucket in path" style endpoints, which are then only option when your bucket name contains dots, for example

https://s3.eu-west-1.amazonaws.com/my.bucket.name

It doesn't appear to make a difference whether I use a custom "key" function.

In this case, for chunked uploads, the "string to sign" sent by Fine Uploader to my signing endpoint has the correct hostname (s3.eu-west-1.amazonaws.com), but the path is just the key, whereas the path in this case should include the bucket name, e.g.

my.bucket.name/key

getCanonicalRequest() in this case includes "method" followed by "end of URL" followed by "querystring".

The endOfUrl is constrcuted as follows:

endOfUrl = requestInfo.key + "?" + endOfUrl;

but if "end of URL" is meant to be "path", this should start with the bucketname, only if the AWS endpoint style doesn't include the bucketname in the hostname.

Does this help?

[batmany13]

I ran into this problem as well, and its because we're using https://s3.eu-central-1.amazonaws.com/<frankfurt-bucket> and in the AWS response, it specifically expects that the "path" include the bucket ie /<frankfurt-bucket>/fine_temp/0256a7b2-bd9e-4b99-9093-9d7ece85efff.mp4 and the "host" should be s3.eu-central-1.amazonaws.com. However, fineuploader sends something different in the "headers" params, with the path as /fine_temp/0256a7b2-bd9e-4b99-9093-9d7ece85efff.mp4 and host as s3.eu-central-1.amazonaws.com/<frankfurt-bucket>. When we changed our signature logic to put the bucket in the path, and removed the bucket from host, it started working.

From AWS

<CanonicalRequest>POST
/<frankfurt-bucket>/fine_temp/0256a7b2-bd9e-4b99-9093-9d7ece85efff.mp4
uploads=
host:s3.eu-central-1.amazonaws.com
x-amz-acl:private
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20180605T210731Z
x-amz-meta-qqfilename:<file>.mp4

host;x-amz-acl;x-amz-content-sha256;x-amz-date;x-amz-meta-qqfilename
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855</CanonicalRequest>

Sent by fineuploader

POST
/fine_temp/0256a7b2-bd9e-4b99-9093-9d7ece85efff.mp4
uploads=
host:s3.eu-central-1.amazonaws.com/<frankfurt-bucket>
x-amz-acl:private
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20180605T210731Z
x-amz-meta-qqfilename:<file>.mp4

host;x-amz-acl;x-amz-content-sha256;x-amz-date;x-amz-meta-qqfilename
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

[truthSeekr]

And what with that approach

POST
/fine_temp/0256a7b2-bd9e-4b99-9093-9d7ece85efff.mp4
uploads=
host:frankfurt-bucket.s3.eu-central-1.amazonaws.com
x-amz-acl:private
x-amz-content-sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
x-amz-date:20180605T210731Z
x-amz-meta-qqfilename:<file>.mp4

host;x-amz-acl;x-amz-content-sha256;x-amz-date;x-amz-meta-qqfilename
e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

Should it work with host like that frankfurt-bucket.s3.eu-central-1.amazonaws.com ????????

[chris-denning]

I see this issue has been closed, but there are 2 other open issues tracking the same problem
#1914
#2010