Monday, May 20, 2024
 Popular · Latest · Hot · Upcoming
137
rated 0 times [  142] [ 5]  / answers: 1 / hits: 11857  / 4 Years ago, thu, june 4, 2020, 12:00:00

I'm trying to connect to a SFTP-Server and list the documents in the /ARCHIVE Folder. Credentials are stored in a .env file. When I run this on my local machine it works and lists the documents.



async function main (event){
let Client = require('ssh2-sftp-client');
let sftp = new Client();
const dotenv = require('dotenv');
dotenv.config();

const ftpOptions = {
host: process.env.FTP_HOST,
port: process.env.FTP_PORT,
username: process.env.FTP_USER,
password: process.env.FTP_PASSWORD,
debug: console.log
}

await sftp.connect(ftpOptions);
let documentList = await sftp.list('/ARCHIVE');
console.log(documentList);
sftp.end();
}


But if I try it in my AWS lambda function, it tries to connect and times out. The env variables are loaded and avaiable before the sftp.connect(ftpOptions) is executed. The logs show that the function is trying to connect to the server, but can't even login with the credentials.



Function Logs:
START RequestId: 20d3c3b7-19d8-49cf-8997-1f2331eb0b6e Version: $LATEST
2020-06-04T08:27:12.837Z 20d3c3b7-19d8-49cf-8997-1f2331eb0b6e INFO Debugging turned on
2020-06-04T08:27:12.860Z 20d3c3b7-19d8-49cf-8997-1f2331eb0b6e INFO DEBUG: Local ident: 'SSH-2.0-ssh2js0.4.10'
2020-06-04T08:27:12.898Z 20d3c3b7-19d8-49cf-8997-1f2331eb0b6e INFO DEBUG: Client: Trying ftp_foo.com on port 22 ...
2020-06-04T08:27:32.929Z 20d3c3b7-19d8-49cf-8997-1f2331eb0b6e INFO CLIENT[sftp]: Connection attempt 1 failed. Trying again.
2020-06-04T08:27:33.931Z 20d3c3b7-19d8-49cf-8997-1f2331eb0b6e INFO DEBUG: Local ident: 'SSH-2.0-ssh2js0.4.10'
2020-06-04T08:27:33.931Z 20d3c3b7-19d8-49cf-8997-1f2331eb0b6e INFO DEBUG: Client: Trying ftp_foo.com on port 22 ...
2020-06-04T08:27:53.951Z 20d3c3b7-19d8-49cf-8997-1f2331eb0b6e INFO CLIENT[sftp]: Exhausted all connection attempts. Giving up
2020-06-04T08:27:53.957Z 20d3c3b7-19d8-49cf-8997-1f2331eb0b6e ERROR Invoke Error {errorType:Error,errorMessage:connect: Timed out while waiting for handshake after 2 attempts,code:ERR_GENERIC_CLIENT,custom:true,stack:[Error: connect: Timed out while waiting for handshake after 2 attempts, at Object.formatError (/var/task/node_modules/ssh2-sftp-client/src/utils.js:62:18), at Client.connectErrorListener (/var/task/node_modules/ssh2-sftp-client/src/index.js:98:21), at Object.onceWrapper (events.js:417:26), at Client.emit (events.js:310:20), at Timeout._onTimeout (/var/task/node_modules/ssh2/lib/client.js:697:14), at listOnTimeout (internal/timers.js:549:17), at processTimers (internal/timers.js:492:7)]}
2020-06-04T08:27:53.959Z 20d3c3b7-19d8-49cf-8997-1f2331eb0b6e ERROR Unhandled Promise Rejection {errorType:Runtime.UnhandledPromiseRejection,errorMessage:Error: end: No SFTP connection available,reason:{errorType:Error,errorMessage:end: No SFTP connection available,code:ERR_NOT_CONNECTED,custom:true,stack:[Error: end: No SFTP connection available, at formatError (/var/task/node_modules/ssh2-sftp-client/src/utils.js:62:18), at Object.haveConnection (/var/task/node_modules/ssh2-sftp-client/src/utils.js:612:20), at /var/task/node_modules/ssh2-sftp-client/src/index.js:1248:19, at new Promise (<anonymous>), at SftpClient.end (/var/task/node_modules/ssh2-sftp-client/src/index.js:1236:12), at downloadPDFs (/var/task/index.js:40:14), at processTicksAndRejections (internal/process/task_queues.js:97:5)]},promise:{},stack:[Runtime.UnhandledPromiseRejection: Error: end: No SFTP connection available, at process.<anonymous> (/var/runtime/index.js:35:15), at process.emit (events.js:310:20), at processPromiseRejections (internal/process/promises.js:209:33), at processTicksAndRejections (internal/process/task_queues.js:98:32)]}
[ERROR] [1591259273978] LAMBDA_RUNTIME Failed to post handler success response. Http response code: 403.



Is there maybe something, that happens on my local machine to provide the connection which I didn't implemented in the lambda function?


More From » node.js

 Answers
9

Solution for SFTP Servers outside of AWS



As mentioned by @Atef before the problem was that the SFTP Server is hosted outside of AWS and only granted access to whitelisted IPs. So I used the following workaround:



Configure a static IP for Lambda function using VPC




  1. Open the VPC Management Console

  2. Create a Elastic IP

  3. Create a VPC with the Wizard (with private and public subnets)

  4. Attach the Elastic IP to the VPC (Elastic IP Allocation ID)

  5. Route your private and public subnet from 0.0.0.0/0 to the elastic IP using the routing table

  6. Create a Internet Gateway and attach it to the VPC

  7. Attach the Lambda function to the VPC


    1. Give the role of your lambda service the permissions AWSLambdaVPCAccessExecutionRole and AmazonEC2FullAccess




You can test this by calling the ipify API from your function and check the ip provided in the HTTP response. It should be the same as your Elastic IP. Now you can whitelist this IP to get access to the SFTP-Server.



For more information also see this article.


[#3582] Tuesday, June 2, 2020, 4 Years  [reply] [flag answer]
Only authorized users can answer the question. Please sign in first, or register a free account.
trayvon

Total Points: 35
Total Questions: 117
Total Answers: 88

Location: Guernsey
Member since Tue, Jul 6, 2021
3 Years ago
trayvon questions
;