SFTP

If you’re new to Unstructured, read this note first.

Before you can create a source connector, you must first sign in to your Unstructured account:

• If you do not already have an Unstructured account, sign up for free. After you sign up, you are automatically signed in to your new Unstructured Let’s Go account, at https://platform.unstructured.io. To sign up for a Business account instead, contact Unstructured Sales, or learn more.
• If you already have an Unstructured Let’s Go, Pay-As-You-Go, or Business SaaS account and are not already signed in, sign in to your account at https://platform.unstructured.io. For other types of Business accounts, see your Unstructured account administrator for sign-in instructions, or email Unstructured Support at [e-mail].

After you sign in, the Unstructured user interface (UI) appears, which you use to get your Unstructured API key, as follows:

1. After you sign in to your Unstructured Let’s Go, Pay-As-You-Go, or Business account, click API Keys on the sidebar.
   

   For a Business account, before you click API Keys, make sure you have selected the organizational workspace you want to create an API key for. Each API key works with one and only one organizational workspace. Learn more.

2. Click Generate API Key.
   

3. Follow the on-screen instructions to finish generating the key.
   

4. Click the Copy icon next to your new key to add the key to your system’s clipboard. If you lose this key, simply return and click the Copy icon again.
   

After you create the source connector, add it along with a destination connector to a workflow. Then run the worklow as a job. To learn how, try out the the notebook Dropbox-To-Pinecone Connector API Quickstart for Unstructured, or watch the two 4-minute video tutorials for the Unstructured Python SDK.

You can also create source connectors with the Unstructured user interface (UI). Learn how.

If you need help, email Unstructured Support at [e-mail].

You are now ready to start creating a source connector! Keep reading to learn how.

Ingest your files into Unstructured from SFTP.

The requirements are as follows.

• The SFTP server hostname, port, username, and password.

  SFTP servers are offered by several vendors. For example, the following video shows how to create and set up an SFTP server by using AWS Transfer Family:

• The directory path to start accessing data from, specified as sftp://<path>/<to>/<directory>.

See also the how-to documentation for SFTP support in popular offerings such as:

• AWS Transfer Family
• Azure Blob Storage
• Couchdrop

To create a SFTP source connector, see the following examples.

Python SDK

import os

from unstructured_client import UnstructuredClient
from unstructured_client.models.operations import CreateSourceRequest
from unstructured_client.models.shared import CreateSourceConnector

with UnstructuredClient(api_key_auth=os.getenv("UNSTRUCTURED_API_KEY")) as client:
    response = client.sources.create_source(
        request=CreateSourceRequest(
            create_source_connector=CreateSourceConnector(
                name="<name>",
                type="sftp",
                config={
                    "host": "<host>",
                    "username": "<username>",
                    "port": "<port>",
                    "recursive": "<true|false>",
                    "password": "<password>"
                }
            )
        )
    )

    print(response.source_connector_information)

Replace the preceding placeholders as follows:

• <name> (required) - A unique name for this connector.
• <host> (required) - The full SFTP server URL (for example, sftp://<hostname>/<path/to/directory>).
• <username> (required) - The username for logging in to the SFTP server.
• <port> - The port number for the SFTP server. The default is 22 if not otherwise specified.
• For recursive (source connector only), true to recursively access files from subdirectories. The default is false (do not recursively access files from subdirectories).
• <password> (required) - The password associated with the username.