Question

3

Download file from url and upload it to AWS S3 without saving - node.js

rated 0 times [ 5] [ 2] / answers: 1 / hits: 39678 / 10 Years ago, wed, march 5, 2014, 12:00:00

I'm writing an application which downloads images from a url and then uploads it to an S3 bucket using the aws-sdk.

Perviously I was just downloading images and saving them to disk like this.

request.head(url, function(err, res, body){



    request(url).pipe(fs.createWriteStream(image_path));



});

And then uploading the images to AWS S3 like this

fs.readFile(image_path, function(err, data){

    s3.client.putObject({

        Bucket: 'myBucket',

        Key: image_path,

        Body: data

        ACL:'public-read'

    }, function(err, resp) {

        if(err){

            console.log(error in s3 put object cb);

        } else { 

            console.log(resp);

            console.log(successfully added image to s3);

        }

    });

});

But I would like to skip the part where I save the image to disk. Is there some way I can pipe the response from request(url) to a variable and then upload that?

Answers

Only authorized users can answer the question. Please sign in first, or register a free account.

luna

Add To Favorites

Follow

Total Points: 698

Total Questions: 114

Total Answers: 93

Location: Israel

Member since Wed, Apr 14, 2021

3 Years ago

luna questions

1 TypeError: Cannot read properties of null (reading 'classList')

Tue, Dec 7, 21, 00:00, 3 Years ago

1 None of these files exist

Sun, Jul 11, 21, 00:00, 3 Years ago

1 How to test failed request with axios / jest

Wed, Jun 30, 21, 00:00, 3 Years ago

1 Howler.js - The AudioContext was not allowed to start. It must be resumed (or created) after a user gesture on the page

Tue, May 25, 21, 00:00, 3 Years ago

1 How to get a single doc with React/Firestore?

Sun, Apr 25, 21, 00:00, 3 Years ago

View All

answered 10 Years ago adilene · Accepted Answer

Here's some javascript that does this nicely:

    var options = {

        uri: uri,

        encoding: null

    };

    request(options, function(error, response, body) {

        if (error || response.statusCode !== 200) { 

            console.log(failed to get image);

            console.log(error);

        } else {

            s3.putObject({

                Body: body,

                Key: path,

                Bucket: 'bucket_name'

            }, function(error, data) { 

                if (error) {

                    console.log(error downloading image to s3);

                } else {

                    console.log(success uploading to s3);

                }

            }); 

        }   

    });