Skip to main content

Transfer Methods

Several Pangea services analyze and process input files and produce output files:

  • File Scan scans input files for malware.
  • Sanitize securely reconstructs input documents and removes potentially dangerous or sensitive content to safeguard the viewer against unintended consequences.
  • Secure Share securely transfers files.

The Pangea APIs for these services support different transfer methods for accepting input files and for providing output files. Depending on your use case, you have several options to choose from. This document explains the available transfer methods, associated parameters, and usage scenarios, accompanied by examples.

Transfer Options Across Services

Pangea API transfer methods offer the choice of uploading input files as multipart/form-data or using various types of presigned URLs. Output files are accessible via presigned URLs, stored in the Secure Share, or provided directly in the service response.

Transfer ParameterFile ScanSecure ShareSanitize
"transfer_method": "post-url"inputinputinput
"transfer_method": "put-url"inputinputinput
"transfer_method": "source-url"inputinputinput
"transfer_method": "share-id"input
"transfer_method": "multipart"inputinput/outputinput
"transfer_method": "dest-url"output: "dest_url"
"share_output": { }output: "dest_share_id" or "dest_url"

Presigned URLs as Transfer Methods

Presigned URLs are secure URLs with a limited duration that provide temporary access, utilizing the security credentials of a third party, enabling the holder to upload or download a file within the specified time frame. Presigned URLs used for uploading files can be either “locked” to specific file properties or “unlocked” and accept any file.

In general, using presigned URLs shifts the file transfer burden onto cloud storage infrastructure, such as AWS, which is robust and has rich support for HTTP transfer options.

Presigned URLs can enable the separation of where an API request is made from where an input file is chosen and uploaded. Your application can leverage presigned URLs by delegating file uploads to its frontend or a third party. Presigned URLs provide options that can help overcome design limitations as well as performance issues within your infrastructure.

Using presigned URLs is a multi-step process that is almost always asynchronous. For example, using File Scan with a presigned URL involves the following requests:

  1. Initiating a File Scan with a presigned URL transfer method for the input file, and receiving the presigned URL for uploading the input file in the response.
  2. Uploading the input file to the presigned URL.
  3. Polling for completion and the results of the File Scan operation.

Learn more about this multi-step process in the Asynchronous API Responses documentation.

Presigned URL Transfer Method Types

When calling a Pangea API that supports presigned URLs there are four related transfer_method parameter values:

post-url

A POST presigned URL ensures the integrity of the uploaded content by locking it to specific file properties like size, sha256 hash, and crc32c signature. When using the post-url transfer method for file processing, you must provide these details. Any file uploaded to a presigned POST URL must match these properties or otherwise will be rejected. While this method offers strong integrity assurance, it might not be suitable for delegating file uploads to a third party or the frontend.

After making a service API call using the post-url transfer method, the response will include result.post_form_data and result.post_url. To upload the file, you must use multipart/form-data serialization and POST the content to the URL provided in result.post_url. The form must include all fields from result.post_form_data, with the file field added at the end.

put-url

A PUT presigned URL allows for the upload of any file contents. With the put-url transfer method, you can request file processing and sign an "unlocked" PUT URL for uploading. The presigned PUT URL can be then shared with the front-end code that selects and provides the file to be processed. This allows for simple separation of backend signing from the frontend file upload. However, unlike the post-url transfer method, put-url lacks robust security provided by locking the presigned URL to specific file content. As a result, presigned PUT URLs allow to upload files of any size within the limits set by the storage provider.

After using the put-url transfer method in a service API call, you'll receive a presigned PUT URL in result.put_url as part of the response. The content of the input file must be included in the body of a PUT request to the URL received in result.put_url.

source-url

A source-url is a caller-specified URL where the Pangea APIs can fetch the contents of the input file. When calling a Pangea API with a transfer_method of source-url, you must also specify a source_url input parameter that provides a URL to the input file. The source URL can be a presigned URL created by the caller, and it will be used to download the content of the input file. The source-url transfer method is useful when you already have a file in your storage and can provide a URL from which Pangea API can fetch the input file—there is no need to transfer it to Pangea with a separate POST or PUT request.

dest-url

A dest-url presigned URL is where the Pangea API will place an output file. When a transfer method of dest-url is specified, the result of the call will contain a dest_url element that contains the URL where the output can be downloaded once the operation is complete. Using dest-url allows you to hand off the dest_url value to another component for downloading the output once the operation is complete.

You can use dest-url as the transfer method with the Secure Share /get and /get_archive APIs. When doing so, the URL for downloading the file from the Secure Share will be provided in result.dest_url within the response.

Note that unless called with the share_output parameter, the Sanitize API also returns a presigned URL in result.dest_url where the sanitized output can be downloaded. This is independent of the transfer method specified in the API call for how input is to be provided for the Sanitize API.

Additional Transfer Methods

share-id

When a service API call requires an input file and supports the share-id transfer method, the input can be referenced from a file stored in the Secure Share. You can use share-id as the input for a Sanitize API call, and then either save the sanitized results in the Secure Share or receive them via a presigned GET URL.

multipart

When a service API call requires an input file, the multipart transfer method allows for uploading the file at the time of the initial request.

When a service API call produces an output file, selecting the multipart transfer method allows for including the file as a multipart/form-data encoded octet-stream in the response.

While the multipart option may seem convenient since it eliminates the need for a separate step in the file transfer process, it does come with certain limitations:

  • This method might not be suitable for use in the frontend of your application or for delegating the file transfer to a third party. Doing so could potentially expose your API token, which is used to authorize the request.

  • When you use the multipart transfer method in the backend, you might need to maintain a scalable infrastructure capable of handling file uploads and downloads.

Transfer Method Parameters

note

The examples provided below won't function properly because they contain sample data specific to a particular environment. If you plan to use them as templates for testing API calls, be sure to replace the sample data with your own values.

post-url

To request a service operation for a file uploaded with a presigned POST URL, you need to specify the following parameters in your initial request:

  • "transfer_method": "post-url"
  • "size": The size (in bytes) of the file. If the upload doesn't match, the call will fail.
  • "crc32c": The CRC32C hash of the file data, which will be verified by the server if provided.
  • "sha256": The hexadecimal-encoded SHA256 hash of the file data, which will be verified by the server if provided.

You can use the Presigned URL Helper to calculate these values.

To get the final result, make the following requests:

  1. Initiate a service operation (such as File Scan).

    Send the request data in JSON format:

    POSTrequest/service/with/post-url
    curl --location 'https://file-scan.aws.pangea.cloud/v1/scan' \
    --header 'Content-Type: application/json' \
    --header "Authorization: Bearer $PANGEA_FILE_SCAN_TOKEN" \
    --data '{
        "transfer_method": "post-url",
        "size": 217104,
        "sha256": "aba4fa35f23e68ee66704252148a5221e18e5c527192467ec63d8add6be2c666",
        "crc32c": "44f66740"
    }'
    

    In the response, the presigned URL returned in result.post_url is locked to the file that matches the provided crc32c, sha256, and size parameter values.

    response/with/presigned/post-url
    {
      "request_id": "prq_pxzbebovkpkm2ght2n7qmbcrnw5w72ax",
      "result": {
          "location": "https://file-scan.aws.us.pangea.cloud/request/prq_pxzbebovkpkm2ght2n7qmbcrnw5w72ax",
          "post_form_data": {
            "key": "20240304/pangea.cloud.xxx_prq_pxzbebovkpkm2ght2n7qmbcrnw5w72ax",
            "policy": "eyJ...ifQ==",
            "x-amz-algorithm": "AWS4-HMAC-SHA256",
            "x-amz-checksum-crc32c": "RPZnQA==",
            "x-amz-credential": "ASIAST5FA55L7SJGHDGN/20240304/us-west-2/s3/aws4_request",
            "x-amz-date": "20240304T010326Z",
            "x-amz-security-token": "IQo...inQ==",
            "x-amz-signature": "be2...919"
          },
          "post_url": "https://pangea-scan-reversinglabs-input.s3.amazonaws.com",
          . . .
      },
      "status": "Accepted",
      "summary": "Your request is in progress. Use 'result, location' below to poll for results. See https://pangea.cloud/docs/api/async?service=file-scan&request_id=prq_pxzbebovkpkm2ght2n7qmbcrnw5w72ax for more information.",
      . . .
    }
    
  2. Upload the file using the presigned POST form.

    Use the multipart/form POST method and the values returned in result.post_form_data to upload the specified file to result.post_url. Provide the reference to your file in a file form field added at the end of the form.

    POSTupload/to/post-url
    curl --location 'https://pangea-scan-reversinglabs-input.s3.amazonaws.com' \
    --form 'key="20240304/pangea.cloud.xxx_prq_pxzbebovkpkm2ght2n7qmbcrnw5w72ax"' \
    --form 'policy="eyJ...ifQ=="' \
    --form 'x-amz-algorithm="AWS4-HMAC-SHA256"' \
    --form 'x-amz-checksum-crc32c="RPZnQA=="' \
    --form 'x-amz-credential="ASI...DGN/20240304/us-west-2/s3/aws4_request"' \
    --form 'x-amz-date="20240304T020328Z"' \
    --form 'x-amz-security-token="IQo...inQ=="' \
    --form 'x-amz-signature="6ef...e5a"' \
    --form "file=@$PATH_TO_FILE"
    
  3. Check the results of the requested operation.

    After the file is uploaded, poll the result.location URL returned in the response to the initial request.

    GETresults/of/service/request
    curl --location 'https://file-scan.aws.us.pangea.cloud/request/prq_pxzbebovkpkm2ght2n7qmbcrnw5w72ax' \
    --header "Authorization: Bearer $PANGEA_FILE_SCAN_TOKEN"
    
    results/of/service/request
    {
      "request_id": "prq_pxzbebovkpkm2ght2n7qmbcrnw5w72ax",
      "status": "Success",
      "summary": "File was scanned",
      "result": {
          "data": {
            "score": 0,
            "verdict": "benign",
            . . .
          }
      },
      . . .
    }
    

put-url

To request a service operation with a presigned PUT URL, you need to specify only the transfer_method parameter in your initial request:

  • "transfer_method": "put-url"

To get the final result, make the following requests:

  1. Initiate a service operation (such as File Scan).

    Send the request data in JSON format:

    POSTrequest/service/with/put-url
    curl --location 'https://file-scan.aws.us.pangea.cloud/v1/scan' \
    --header "Authorization: Bearer $PANGEA_FILE_SCAN_TOKEN" \
    --header 'Content-Type: application/json' \
    --data '{"transfer_method":"put-url"}'
    

    The URL returned in result.put_url can be used to upload any file that the storage will accept.

    response/with/presigned/put-url
    {
      "request_id": "prq_hc6addbdgbel6pptcwin67fon2fnib2a",
      "result": {
        "location": "https://file-scan.aws.us.pangea.cloud/request/prq_hc6addbdgbel6pptcwin67fon2fnib2a",
        "put_url": "https://pangea-scan-reversinglabs-input.s3.us-west-2.amazonaws.com/20240304/pangea.cloud.xxx_prq_hc6addbdgbel6pptcwin67fon2fnib2a?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=ASI...NVA%2F20240304%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20240304T063957Z&X-Amz-Expires=3600&X-Amz-Security-Token=IQo...Cpg%3D%3D&X-Amz-SignedHeaders=host&X-Amz-Signature=562...566",
        . . .
      },
      "status": "Accepted",
      "summary": "Your request is in progress. Use 'result, location' below to poll for results. See https://pangea.cloud/docs/api/async?service=file-scan&request_id=prq_hc6addbdgbel6pptcwin67fon2fnib2a for more information.",
      . . .
    }
    
  2. Upload the file using the presigned PUT URL returned in result.put_url.

    Using the presigned PUT URL, you can send the file content unencoded, as binary, and specify the content type in a header.

    PUTupload/to/put-url
    curl --location --request PUT 'https://pangea-scan-reversinglabs-input.s3.us-west-2.amazonaws.com/20240304/pangea.cloud.xxx_prq_hc6addbdgbel6pptcwin67fon2fnib2a?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=ASI...STJ%2F20240304%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20240304T073145Z&X-Amz-Expires=3600&X-Amz-Security-Token=IQo...gkw%3D%3D&X-Amz-SignedHeaders=host&X-Amz-Signature=34b5...778' \
    --header 'Content-Type: application/octet-stream' \
    --data-binary "@$PATH_TO_FILE"
    

    You can also encode the file content as multipart/form-data, for example:

    --form "file=@$PATH_TO_FILE;type=application/octet-stream"

  3. Check the results of the requested operation.

    After the file is uploaded, poll the result.location URL returned in the response to the initial request.

    GETresults/of/service/request
    curl --location 'https://file-scan.aws.us.pangea.cloud/request/prq_hc6addbdgbel6pptcwin67fon2fnib2a' \
    --header "Authorization: Bearer $PANGEA_FILE_SCAN_TOKEN"
    
    results/of/service/request
    {
      "request_id": "prq_hc6addbdgbel6pptcwin67fon2fnib2a",
      "status": "Success",
      "summary": "File was scanned",
      "result": {
          "data": {
            "score": 0,
            "verdict": "benign",
            . . .
          }
      },
      . . .
    }
    

source-url

To request a service operation for a file that can be downloaded from a caller-specified URL, you need to specify the following parameters in your initial request:

  • "transfer_method": "source-url"
  • "source_url": The URL of the input file to be downloaded by the API.

To get the final result, make the following requests:

  1. Initiate a service operation (such as File Scan).

    Send the request data in JSON format:

    POSTprovide/file/with/source-url
    curl --location 'https://file-scan.aws.us.pangea.cloud/v1/scan' \
    --header 'Content-Type: application/json' \
    --header "Authorization: Bearer $PANGEA_FILE_SCAN_TOKEN" \
    --data '{
        "transfer_method": "source-url",
        "source_url": "https://my-scan-input.s3.us-west-2.amazonaws.com/samples/my_tiny.pdf?..."
    }'
    
    response/with/results/location
    {
      "request_id": "prq_7ciapytasjqtha7qtuqycl6bebjc6vo4",
      "result": {
          "location": "https://file-scan.aws.us.pangea.cloud/request/prq_7ciapytasjqtha7qtuqycl6bebjc6vo4",
          . . .
      },
      "status": "Accepted",
      "summary": "Your request is in progress. Use 'result, location' below to poll for results. See https://pangea.cloud/docs/api/async?service=file-scan&request_id=prq_7ciapytasjqtha7qtuqycl6bebjc6vo4 for more information."
    }
    
  2. Check the results of the requested operation.

    With the source-url transfer method, you are telling Pangea where to fetch the input. Hence, there is no file upload step, and you can immediately start polling for the results of the requested operation using the result.location URL returned from the initial request.

    GETresults/of/service/request
    curl --location 'https://file-scan.aws.us.pangea.cloud/request/prq_7ciapytasjqtha7qtuqycl6bebjc6vo4' \
    --header "Authorization: Bearer $PANGEA_FILE_SCAN_TOKEN"
    
    results/of/service/request
    {
      "request_id": "prq_7ciapytasjqtha7qtuqycl6bebjc6vo4",
      "status": "Success",
      "summary": "File was scanned",
      "result": {
          "data": {
            "score": 0,
            "verdict": "benign",
            . . .
          }
      },
      . . .
    }
    

dest-url

When requesting a download from the Secure Share using its /get or /get_archive API endpoints, you have the option to select dest-url as the transfer_method. This will provide you with a URL in result.dest-url within the response, from which the output file can be downloaded.

Include the following parameters in your request:

  • "transfer_method": "dest-url"
  • "id" or "path": Provide either the ID or path to a file or folder within the Secure Share.

Send the request data in JSON format:

POSTrequest/file/with/dest-url
curl --location 'https://share.aws.us.pangea.cloud/v1beta/get' \
--header 'Content-Type: application/json' \
--header "Authorization: Bearer $PANGEA_SECURE_SHARE_TOKEN" \
--data '{
    "transfer_method": "dest-url",
    "id": "pos_pwwejxtwoaamkvjqy4gzscetchexoglp"
}'

The API returns a presigned GET URL in result.dest_url where the file from the Secure Share can be downloaded.

response/with/dest_url
{
  "request_id": "prq_7dmcm64tyzxm7hlsbfhz33l577ifhyzm",
  "status": "Success",
  "summary": "Object get successful.",
  "result": {
    "dest_url": "https://pangea-share.s3.us-west-2.amazonaws.com/share-tmp/prq_7dmcm64tyzxm7hlsbfhz33l577ifhyzm?...",
    "object": {
      "folder": "/",
      "id": "pos_pwwejxtwoaamkvjqy4gzscetchexoglp",
      . . .
    }
  }
  . . .
}

multipart

The multipart transfer method is an option that does not leverage presigned URLs. It can be utilized for both providing an input file and receiving an output file, as outlined below:

  • When the API call requires an input file, the caller uploads the file encoded as multipart/form-data using the POST method as part of the initial request.

    To get the final result, make the following requests:

    1. Initiate a service operation (such as File Scan).

      POSTmultipart/form-data
      curl --location 'https://file-scan.aws.us.pangea.cloud/v1/scan' \
      --header "Authorization: Bearer $PANGEA_FILE_SCAN_TOKEN" \
      --form 'request="{\"transfer_method\":\"multipart\"}";type=application/json' \
      --form "file=@$PATH_TO_FILE"
      
      response/with/results/location
      {
        "request_id": "prq_3f2hlqm3nkd23tuqpfcsxymhkskyvlgp",
        "result": {
          "location": "https://file-scan.aws.us.pangea.cloud/request/prq_3f2hlqm3nkd23tuqpfcsxymhkskyvlgp",
          . . .
        },
        "status": "Accepted",
        "summary": "Your request is in progress. Use 'result, location' below to poll for results. See https://pangea.cloud/docs/api/async?service=file-scan&request_id=prq_3f2hlqm3nkd23tuqpfcsxymhkskyvlgp for more information."
      }
      
    2. Check the results of the requested operation.

      The file to be processed is uploaded as a part of the initial request, and you can immediately start checking the results of the requested service operation using the result.location URL returned in the response.

      GETresults/of/service/request
      curl --location 'https://file-scan.aws.us.pangea.cloud/request/prq_3f2hlqm3nkd23tuqpfcsxymhkskyvlgp' \
      --header "Authorization: Bearer $PANGEA_FILE_SCAN_TOKEN"
      
      results/of/service/request
      {
        "request_id": "prq_lnzw6zpns2z6e4xyyzrb26ssjod7hh7o",
        "status": "Success",
        "summary": "File was scanned",
        "result": {
          "data": {
            "verdict": "benign",
            . . .
          }
        },
        . . .
      }
      
  • In cases where the API call generates an output file and the transfer_method is multipart, the 200 response will contain the file encoded as a multipart/form-data octet-stream.

    When requesting a download from the Secure Share using its /get or /get_archive API endpoints, you have the option to include the file content in the response. This can be done by selecting the multipart transfer method option.

    In your request, include the following parameters:

    • "transfer_method": "multipart"
    • "id" or "path": Provide either the ID or path to a file or folder within the Secure Share.

    Send the request data in JSON format:

    POSTrequest/file/with/dest-url
    curl --location 'https://share.aws.us.pangea.cloud/v1beta/get' \
    --header 'Content-Type: application/json' \
    --header "Authorization: Bearer $PANGEA_SECURE_SHARE_TOKEN" \
    --data '{
        "transfer_method": "multipart",
        "id": "pos_bbydypfioc2vta4zf7jdryw6a6j3p5ne"
    }' --remote-name --remote-header-name
    

    The file content is included in the response and can be saved in the local file system.

share-id

The Sanitize service can be used against a file saved in the Secure Share. To make the API call, you need to specify the following parameters in your initial request:

  • "transfer_method": "share-id"
  • "share_id": The ID of the file saved in the Secure Share.

A successful response from the Sanitize service will contain a presigned GET URL in result.dest_url, which you can use to download the sanitized output.

To get the final result, make the following requests:

  1. Request sanitization.

    Send the request data in JSON format:

    POSTrequest/sanitize/with/share-id
    curl --location 'https://sanitize.aws.us.pangea.cloud/v1beta/sanitize' \
    --header 'Content-Type: application/json' \
    --header "Authorization: Bearer $PANGEA_SANITIZE_TOKEN" \
    --data '{
        "transfer_method": "share-id",
        "share_id": "pos_jxap2xe7guejee3udi5bc4mu4tbdol5w"
    }'
    

    After making a call to the Sanitize service, you'll receive an Asynchronous Response. This response contains a GET URL in result.location, which you can use to track the status of your request.

    request/sanitize/with/share-id
    {
        "request_id": "prq_xao2vkxsxbk37fgf5zmdwh3vgiterc34",
        "status": "Accepted",
        "summary": "Your request is in progress. Use 'result, location' below to poll for results. See https://pangea.cloud/docs/api/async?service=sanitize&request_id=prq_xao2vkxsxbk37fgf5zmdwh3vgiterc34 for more information.",
        "result": {
            "location": "https://sanitize.aws.us.pangea.cloud/request/prq_xao2vkxsxbk37fgf5zmdwh3vgiterc34",
            . . .
        },
        . . .
    }
    
  2. Check the results of the requested sanitization.

    GETresults/of/sanitize
    curl --location 'https://sanitize.aws.us.pangea.cloud/request/prq_xao2vkxsxbk37fgf5zmdwh3vgiterc34' \
    --header "Authorization: Bearer $PANGEA_SANITIZE_TOKEN"
    

    Use the presigned GET URL in result.dest_url within the response to download the sanitized output.

    results/of/sanitize
    {
      "request_id": "prq_xao2vkxsxbk37fgf5zmdwh3vgiterc34",
      "result": {
          "dest_url": "https://pangea-sanitize-input.s3.us-west-2.amazonaws.com/2024030423/prq_64tjspdh4yxxownpm2ap4qb4rbxuedeo/sanitized.my_tiny.pdf?...",
          . . .
      },
      "status": "Success",
      "summary": "Successfully completed the request.  The file download link is valid for 24h0m0s."
    }
    

Additional Parameters for the Sanitize Service

share_output

You can choose how the results of a Sanitize API call are delivered by optionally specifying an additional parameter, share_output.

For a Sanitize API service call, an input file can be provided using different transfer_method options discussed earlier:

  • "source-url"
  • "put-url"
  • "post-url"
  • "share-id"
  • "multipart"

You have two options for receiving the results of a Sanitize API call:

  • If you omit the optional share_output parameter in your initial request, the successful response from the Sanitize service will contain a presigned GET URL in result.dest_url, which you can use to download the sanitized output.

    For example:

    1. Request sanitization.

      POSTsanitize/file/at/source-url
      curl --location 'https://sanitize.aws.us.pangea.cloud/v1beta/sanitize' \
      --header 'Content-Type: application/json' \
      --header "Authorization: Bearer $PANGEA_SANITIZE_TOKEN" \
      --data '{
          "transfer_method": "source-url",
          "source_url": "https://my-scan-input.s3.us-west-2.amazonaws.com/samples/my_tiny.pdf?..."
      }'
      

      After making a call to the Sanitize service, you'll receive an Asynchronous Response. This response contains a GET URL in result.location, which you can use to track the status of your request.

      response/with/results/location
      {
        "request_id": "prq_64tjspdh4yxxownpm2ap4qb4rbxuedeo",
        "status": "Accepted",
        "summary": "Your request is in progress. Use 'result, location' below to poll for results. See https://pangea.cloud/docs/api/async?service=sanitize&request_id=prq_64tjspdh4yxxownpm2ap4qb4rbxuedeo for more information.",
        "result": {
            "location": "https://sanitize.aws.us.pangea.cloud/request/prq_64tjspdh4yxxownpm2ap4qb4rbxuedeo",
            . . .
        },
        . . .
      }
      
    2. Check the results of the requested sanitization.

      GETresults/of/sanitize
      curl --location 'https://sanitize.aws.us.pangea.cloud/request/prq_64tjspdh4yxxownpm2ap4qb4rbxuedeo' \
      --header "Authorization: Bearer $PANGEA_SANITIZE_TOKEN"
      

      Use the presigned GET URL in result.dest_url to download the sanitized output.

      results/of/sanitize
      {
        "request_id": "prq_64tjspdh4yxxownpm2ap4qb4rbxuedeo",
        "result": {
            "dest_url": "https://pangea-sanitize-input.s3.us-west-2.amazonaws.com/2024030423/prq_64tjspdh4yxxownpm2ap4qb4rbxuedeo/sanitized.my_tiny.pdf?...",
            . . .
        },
        "status": "Success",
        "summary": "Successfully completed the request.  The file download link is valid for 24h0m0s."
      }
      
  • Enabling share_output in your initial request saves the sanitized output in the Secure Share.

    For example:

    1. Request sanitization.

      POSTsanitize/file/at/source-url
      curl --location 'https://sanitize.aws.us.pangea.cloud/v1beta/sanitize' \
      --header 'Content-Type: application/json' \
      --header "Authorization: Bearer $PANGEA_SANITIZE_TOKEN" \
      --data '{
        "transfer_method": "source-url",
        "source_url": "https://pangea-sanitize-input.s3.us-west-2.amazonaws.com/samples/redact_tiny.pdf?...",
        "share_output": {
            "enabled": true,
            "output_folder": "/"
        }
      }'
      

      Note that if you specify a non-existent "output_folder" location, the Secure Share will automatically create it for you.

      In the response, you'll receive a GET URL in the result.location. You can use this URL to check the status of the call and its eventual results.

      response/with/results/location
      {
        "request_id": "prq_zrdj2aggcspg6nslzlk7im63s577o34z",
        "result": {
          "location": "https://sanitize.aws.us.pangea.cloud/request/prq_zrdj2aggcspg6nslzlk7im63s577o34z",
          . . .
        },
        "status": "Accepted",
        "summary": "Your request is in progress. Use 'result, location' below to poll for results. See https://pangea.cloud/docs/api/async?service=sanitize&request_id=prq_zrdj2aggcspg6nslzlk7im63s577o34z for more information.",
        . . .
      }
      
    2. Check the results of the requested sanitization.

      GETresults/of/sanitize
      curl --location 'https://sanitize.aws.us.pangea.cloud/request/prq_zrdj2aggcspg6nslzlk7im63s577o34z' \
      --header "Authorization: Bearer $PANGEA_SANITIZE_TOKEN"
      

      If the call is successful, result.dest_share_id will contain the ID of the file saved in the Secure Share.

      results/of/sanitize
      {
        "request_id": "prq_zrdj2aggcspg6nslzlk7im63s577o34z",
        "status": "Success",
        "summary": "Successfully completed the request.  The Sanitized file sanitized.Asynchronous API Responses Pangea.pdf can be found in the Secure Share under folder: /.",
        "result": {
          "dest_share_id": "pos_pp2l24fj7kcdafmyqtztd6oeoofpmeid",
          . . .
        },
        . . .
      }
      

Example: Using transfer method post-url with the File Scan service

The Python code snippet below uses the post-url transfer_method and does the following:

  1. Calls the file_scan_endpoint (File Scan API) with transfer_method of post-url.

    • The crc32c (hex-encoded), sha256 (hex-encoded), and size would need to be calculated from the file to be uploaded and set in these parameters (not shown). You can use the Presigned URL Helper to calculate these values.

    • You also need to supply your File Scan authorization token represented by SCAN_AUTH_TOKEN in the code snippet below.

  2. Extracts final_scan_result_url containing the request_id, the post_url, and the post_form_data from the file_scan_response.

  3. The post_form_data are multipart/form-data encoded as fields into the body of the POST request to the post_url.

  4. The contents of the file to be scanned are multipart/form-data encoded into the body as the final field of the POST request to the post_url.

  5. The file is sent as a POST to the post_url for File Scan to receive and scan.

  6. The final_scan_result_url is used to poll for a final File Scan result.

note

The following code snippet is only intended to be used as an illustration and does not contain all of the necessary code to properly make Presigned URL API calls.

POST/v1/File
import requests

# (SEE 1 ABOVE)
# Prepare the body of the initial POST
request_body = {
  "verbose": true,
  "raw": true,
  "provider": "crowdstrike",
  "transfer_method": "post-url",
  "crc32c": "hex encoded crc_value_here",
  "sha256": "hex encoded sha_value_here",
  "size": size_value_here
}

# (SEE 1 ABOVE)
# Pangea authorization
headers = {
    "Authorization": "Bearer " + SCAN_AUTH_TOKEN,
}

# (SEE 1 ABOVE)
# POST to the Pangea File Scan API endpoint
file_scan_endpoint = "https://file-scan.aws.us.pangea.cloud/v1/scan"
file_scan_response = requests.Request("POST", file_scan_endpoint, headers=headers, json=request_body)
session = requests.session()
session.send(file_scan_response)

# (SEE 2 ABOVE)
# Get the response and the values needed to post to the upload URL
status_code = file_scan_response.status_code
json_response = file_scan_response.json()
final_scan_result_url = json_response["result"]["location"]
post_form_data = json_response["result"]["post_form_data"]
post_url = json_response["result"]["post_url"]

# (SEE 3 ABOVE)
# Create a form-data encoded body for the upload that contains the post_form_data from the API call
body = []
for k, v in post_form_data.items():
    body.append((k, v))

# (SEE 4 ABOVE)
# Add the file to be scanned to the POST
files = {
    "file": ("filename.ext", file.open("<path to the file>"), "application/octet-stream"),
}

# (SEE 5 ABOVE)
# Upload the file to the post_url for File Scan to scan
post_url_upload_response = requests.post(post_url, data=body, files=files, verify=verify)

# (SEE 6 ABOVE)
# Poll for the File Scan result using the final_scan_result_url
while True:
    result_response = requests.get(final_scan_result_url, headers=headers, verify=verify)

    if result_response.status_code == 200:
        result_data = result_response.json()
        if result_data["status"] == "Completed":
            print("Scan completed successfully")
            # Process results if needed
            break
        elif result_data["status"] == "InProgress":
            print("Scan still in progress. Polling...")
            time.sleep(10)  # Sleep for a while before polling again
        else:
            print("Scan status: " + result_data["status"])
            break
    else:
        print("Error: Unable to retrieve scan result")
        break

Presigned URL Helper

Upload a file to this helper to get the following data:

  1. The SHA-256 hash of the file.
  2. The CRC32C hash of the file.
  3. The size of the file (in bytes).