Transfer Methods
Several Pangea services analyze and process input files and produce output files:
- File Scan scans input files for malware.
- Sanitize securely reconstructs input documents and removes potentially dangerous or sensitive content to safeguard the viewer against unintended consequences.
- Secure Share securely transfers files.
The Pangea APIs for these services support different transfer methods for accepting input files and for providing output files. Depending on your use case, you have several options to choose from. This document explains the available transfer methods, associated parameters, and usage scenarios, accompanied by examples.
Transfer Options Across Services
Pangea API transfer methods offer the choice of uploading input files as multipart/form-data
or using various types of presigned URLs. Output files are accessible via presigned URLs, stored in the Secure Share, or provided directly in the service response.
Transfer Parameter | File Scan | Secure Share | Sanitize |
---|---|---|---|
"transfer_method": "post-url" | input | input | input |
"transfer_method": "put-url" | input | input | input |
"transfer_method": "source-url" | input | input | input |
"transfer_method": "share-id" | input | ||
"transfer_method": "multipart" | input | input/output | input |
"transfer_method": "dest-url" | output: "dest_url" | ||
"share_output": { } | output: "dest_share_id" or "dest_url" |
Presigned URLs as Transfer Methods
Presigned URLs are secure URLs with a limited duration that provide temporary access, utilizing the security credentials of a third party, enabling the holder to upload or download a file within the specified time frame. Presigned URLs used for uploading files can be either “locked” to specific file properties or “unlocked” and accept any file.
In general, using presigned URLs shifts the file transfer burden onto cloud storage infrastructure, such as AWS, which is robust and has rich support for HTTP transfer options.
Presigned URLs can enable the separation of where an API request is made from where an input file is chosen and uploaded. Your application can leverage presigned URLs by delegating file uploads to its frontend or a third party. Presigned URLs provide options that can help overcome design limitations as well as performance issues within your infrastructure.
Using presigned URLs is a multi-step process that is almost always asynchronous. For example, using File Scan with a presigned URL involves the following requests:
- Initiating a File Scan with a presigned URL transfer method for the input file, and receiving the presigned URL for uploading the input file in the response.
- Uploading the input file to the presigned URL.
- Polling for completion and the results of the File Scan operation.
Learn more about this multi-step process in the Asynchronous API Responses documentation.
Presigned URL Transfer Method Types
When calling a Pangea API that supports presigned URLs there are four related transfer_method
parameter values:
post-url
A POST presigned URL ensures the integrity of the uploaded content by locking it to specific file properties like size
, sha256
hash, and crc32c
signature. When using the post-url
transfer method for file processing, you must provide these details. Any file uploaded to a presigned POST URL must match these properties or otherwise will be rejected. While this method offers strong integrity assurance, it might not be suitable for delegating file uploads to a third party or the frontend.
After making a service API call using the post-url
transfer method, the response will include result.post_form_data
and result.post_url
. To upload the file, you must use multipart/form-data
serialization and POST the content to the URL provided in result.post_url
. The form must include all fields from result.post_form_data
, with the file
field added at the end.
put-url
A PUT presigned URL allows for the upload of any file contents. With the put-url
transfer method, you can request file processing and sign an "unlocked" PUT URL for uploading. The presigned PUT URL can be then shared with the front-end code that selects and provides the file to be processed. This allows for simple separation of backend signing from the frontend file upload. However, unlike the post-url
transfer method, put-url
lacks robust security provided by locking the presigned URL to specific file content. As a result, presigned PUT URLs allow to upload files of any size within the limits set by the storage provider.
After using the put-url
transfer method in a service API call, you'll receive a presigned PUT URL in result.put_url
as part of the response. The content of the input file must be included in the body of a PUT request to the URL received in result.put_url
.
source-url
A source-url
is a caller-specified URL where the Pangea APIs can fetch the contents of the input file. When calling a Pangea API with a transfer_method
of source-url
, you must also specify a source_url
input parameter that provides a URL to the input file. The source URL can be a presigned URL created by the caller, and it will be used to download the content of the input file. The source-url
transfer method is useful when you already have a file in your storage and can provide a URL from which Pangea API can fetch the input file—there is no need to transfer it to Pangea with a separate POST or PUT request.
dest-url
A dest-url
presigned URL is where the Pangea API will place an output file. When a transfer method of dest-url
is specified, the result of the call will contain a dest_url
element that contains the URL where the output can be downloaded once the operation is complete. Using dest-url
allows you to hand off the dest_url
value to another component for downloading the output once the operation is complete.
You can use dest-url
as the transfer method with the Secure Share /get and /get_archive APIs. When doing so, the URL for downloading the file from the Secure Share will be provided in result.dest_url
within the response.
Note that unless called with the share_output
parameter, the Sanitize API also returns a presigned URL in result.dest_url
where the sanitized output can be downloaded. This is independent of the transfer method specified in the API call for how input is to be provided for the Sanitize API.
Additional Transfer Methods
share-id
When a service API call requires an input file and supports the share-id
transfer method, the input can be referenced from a file stored in the Secure Share. You can use share-id
as the input for a Sanitize API call, and then either save the sanitized results in the Secure Share or receive them via a presigned GET URL.
multipart
When a service API call requires an input file, the multipart
transfer method allows for uploading the file at the time of the initial request.
When a service API call produces an output file, selecting the multipart
transfer method allows for including the file as a multipart/form-data
encoded octet-stream
in the response.
While the multipart
option may seem convenient since it eliminates the need for a separate step in the file transfer process, it does come with certain limitations:
-
This method might not be suitable for use in the frontend of your application or for delegating the file transfer to a third party. Doing so could potentially expose your API token, which is used to authorize the request.
-
When you use the
multipart
transfer method in the backend, you might need to maintain a scalable infrastructure capable of handling file uploads and downloads.
Transfer Method Parameters
The examples provided below won't function properly because they contain sample data specific to a particular environment. If you plan to use them as templates for testing API calls, be sure to replace the sample data with your own values.
post-url
To request a service operation for a file uploaded with a presigned POST URL, you need to specify the following parameters in your initial request:
- "transfer_method": "post-url"
- "size": The size (in bytes) of the file. If the upload doesn't match, the call will fail.
- "crc32c": The CRC32C hash of the file data, which will be verified by the server if provided.
- "sha256": The hexadecimal-encoded SHA256 hash of the file data, which will be verified by the server if provided.
You can use the Presigned URL Helper to calculate these values.
To get the final result, make the following requests:
-
Initiate a service operation (such as File Scan).
Send the request data in JSON format:
POSTrequest/service/with/post-urlcURLcurl --location 'https://file-scan.aws.pangea.cloud/v1/scan' \ --header 'Content-Type: application/json' \ --header "Authorization: Bearer $PANGEA_FILE_SCAN_TOKEN" \ --data '{ "transfer_method": "post-url", "size": 217104, "sha256": "aba4fa35f23e68ee66704252148a5221e18e5c527192467ec63d8add6be2c666", "crc32c": "44f66740" }'
In the response, the presigned URL returned in
result.post_url
is locked to the file that matches the providedcrc32c
,sha256
, andsize
parameter values.response/with/presigned/post-urljson{ "request_id": "prq_pxzbebovkpkm2ght2n7qmbcrnw5w72ax", "result": { "location": "https://file-scan.aws.us.pangea.cloud/request/prq_pxzbebovkpkm2ght2n7qmbcrnw5w72ax", "post_form_data": { "key": "20240304/pangea.cloud.xxx_prq_pxzbebovkpkm2ght2n7qmbcrnw5w72ax", "policy": "eyJ...ifQ==", "x-amz-algorithm": "AWS4-HMAC-SHA256", "x-amz-checksum-crc32c": "RPZnQA==", "x-amz-credential": "ASIAST5FA55L7SJGHDGN/20240304/us-west-2/s3/aws4_request", "x-amz-date": "20240304T010326Z", "x-amz-security-token": "IQo...inQ==", "x-amz-signature": "be2...919" }, "post_url": "https://pangea-scan-reversinglabs-input.s3.amazonaws.com", . . . }, "status": "Accepted", "summary": "Your request is in progress. Use 'result, location' below to poll for results. See https://pangea.cloud/docs/api/async?service=file-scan&request_id=prq_pxzbebovkpkm2ght2n7qmbcrnw5w72ax for more information.", . . . }
-
Upload the file using the presigned POST form.
Use the
multipart/form
POST method and the values returned inresult.post_form_data
to upload the specified file toresult.post_url
. Provide the reference to your file in afile
form field added at the end of the form.POSTupload/to/post-urlcURLcurl --location 'https://pangea-scan-reversinglabs-input.s3.amazonaws.com' \ --form 'key="20240304/pangea.cloud.xxx_prq_pxzbebovkpkm2ght2n7qmbcrnw5w72ax"' \ --form 'policy="eyJ...ifQ=="' \ --form 'x-amz-algorithm="AWS4-HMAC-SHA256"' \ --form 'x-amz-checksum-crc32c="RPZnQA=="' \ --form 'x-amz-credential="ASI...DGN/20240304/us-west-2/s3/aws4_request"' \ --form 'x-amz-date="20240304T020328Z"' \ --form 'x-amz-security-token="IQo...inQ=="' \ --form 'x-amz-signature="6ef...e5a"' \ --form "file=@$PATH_TO_FILE"
-
Check the results of the requested operation.
After the file is uploaded, poll the
result.location
URL returned in the response to the initial request.GETresults/of/service/requestcURLcurl --location 'https://file-scan.aws.us.pangea.cloud/request/prq_pxzbebovkpkm2ght2n7qmbcrnw5w72ax' \ --header "Authorization: Bearer $PANGEA_FILE_SCAN_TOKEN"
results/of/service/requestjson{ "request_id": "prq_pxzbebovkpkm2ght2n7qmbcrnw5w72ax", "status": "Success", "summary": "File was scanned", "result": { "data": { "score": 0, "verdict": "benign", . . . } }, . . . }
put-url
To request a service operation with a presigned PUT URL, you need to specify only the transfer_method
parameter in your initial request:
- "transfer_method": "put-url"
To get the final result, make the following requests:
-
Initiate a service operation (such as File Scan).
Send the request data in JSON format:
POSTrequest/service/with/put-urlcURLcurl --location 'https://file-scan.aws.us.pangea.cloud/v1/scan' \ --header "Authorization: Bearer $PANGEA_FILE_SCAN_TOKEN" \ --header 'Content-Type: application/json' \ --data '{"transfer_method":"put-url"}'
The URL returned in
result.put_url
can be used to upload any file that the storage will accept.response/with/presigned/put-urljson{ "request_id": "prq_hc6addbdgbel6pptcwin67fon2fnib2a", "result": { "location": "https://file-scan.aws.us.pangea.cloud/request/prq_hc6addbdgbel6pptcwin67fon2fnib2a", "put_url": "https://pangea-scan-reversinglabs-input.s3.us-west-2.amazonaws.com/20240304/pangea.cloud.xxx_prq_hc6addbdgbel6pptcwin67fon2fnib2a?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=ASI...NVA%2F20240304%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20240304T063957Z&X-Amz-Expires=3600&X-Amz-Security-Token=IQo...Cpg%3D%3D&X-Amz-SignedHeaders=host&X-Amz-Signature=562...566", . . . }, "status": "Accepted", "summary": "Your request is in progress. Use 'result, location' below to poll for results. See https://pangea.cloud/docs/api/async?service=file-scan&request_id=prq_hc6addbdgbel6pptcwin67fon2fnib2a for more information.", . . . }
-
Upload the file using the presigned PUT URL returned in
result.put_url
.Using the presigned PUT URL, you can send the file content unencoded, as binary, and specify the content type in a header.
PUTupload/to/put-urlcURLcurl --location --request PUT 'https://pangea-scan-reversinglabs-input.s3.us-west-2.amazonaws.com/20240304/pangea.cloud.xxx_prq_hc6addbdgbel6pptcwin67fon2fnib2a?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=ASI...STJ%2F20240304%2Fus-west-2%2Fs3%2Faws4_request&X-Amz-Date=20240304T073145Z&X-Amz-Expires=3600&X-Amz-Security-Token=IQo...gkw%3D%3D&X-Amz-SignedHeaders=host&X-Amz-Signature=34b5...778' \ --header 'Content-Type: application/octet-stream' \ --data-binary "@$PATH_TO_FILE"
You can also encode the file content as
multipart/form-data
, for example:--form "file=@$PATH_TO_FILE;type=application/octet-stream"
-
Check the results of the requested operation.
After the file is uploaded, poll the
result.location
URL returned in the response to the initial request.GETresults/of/service/requestcURLcurl --location 'https://file-scan.aws.us.pangea.cloud/request/prq_hc6addbdgbel6pptcwin67fon2fnib2a' \ --header "Authorization: Bearer $PANGEA_FILE_SCAN_TOKEN"
results/of/service/requestjson{ "request_id": "prq_hc6addbdgbel6pptcwin67fon2fnib2a", "status": "Success", "summary": "File was scanned", "result": { "data": { "score": 0, "verdict": "benign", . . . } }, . . . }
source-url
To request a service operation for a file that can be downloaded from a caller-specified URL, you need to specify the following parameters in your initial request:
- "transfer_method": "source-url"
- "source_url": The URL of the input file to be downloaded by the API.
To get the final result, make the following requests:
-
Initiate a service operation (such as File Scan).
Send the request data in JSON format:
POSTprovide/file/with/source-urlcURLcurl --location 'https://file-scan.aws.us.pangea.cloud/v1/scan' \ --header 'Content-Type: application/json' \ --header "Authorization: Bearer $PANGEA_FILE_SCAN_TOKEN" \ --data '{ "transfer_method": "source-url", "source_url": "https://my-scan-input.s3.us-west-2.amazonaws.com/samples/my_tiny.pdf?..." }'
response/with/results/locationjson{ "request_id": "prq_7ciapytasjqtha7qtuqycl6bebjc6vo4", "result": { "location": "https://file-scan.aws.us.pangea.cloud/request/prq_7ciapytasjqtha7qtuqycl6bebjc6vo4", . . . }, "status": "Accepted", "summary": "Your request is in progress. Use 'result, location' below to poll for results. See https://pangea.cloud/docs/api/async?service=file-scan&request_id=prq_7ciapytasjqtha7qtuqycl6bebjc6vo4 for more information." }
-
Check the results of the requested operation.
With the
source-url
transfer method, you are telling Pangea where to fetch the input. Hence, there is no file upload step, and you can immediately start polling for the results of the requested operation using theresult.location
URL returned from the initial request.GETresults/of/service/requestcURLcurl --location 'https://file-scan.aws.us.pangea.cloud/request/prq_7ciapytasjqtha7qtuqycl6bebjc6vo4' \ --header "Authorization: Bearer $PANGEA_FILE_SCAN_TOKEN"
results/of/service/requestjson{ "request_id": "prq_7ciapytasjqtha7qtuqycl6bebjc6vo4", "status": "Success", "summary": "File was scanned", "result": { "data": { "score": 0, "verdict": "benign", . . . } }, . . . }
dest-url
When requesting a download from the Secure Share using its /get or /get_archive API endpoints, you have the option to select dest-url
as the transfer_method
. This will provide you with a URL in result.dest-url
within the response, from which the output file can be downloaded.
Include the following parameters in your request:
- "transfer_method": "dest-url"
- "id" or "path": Provide either the ID or path to a file or folder within the Secure Share.
Send the request data in JSON format:
curl --location 'https://share.aws.us.pangea.cloud/v1/get' \
--header 'Content-Type: application/json' \
--header "Authorization: Bearer $PANGEA_SECURE_SHARE_TOKEN" \
--data '{
"transfer_method": "dest-url",
"id": "pos_pwwejxtwoaamkvjqy4gzscetchexoglp"
}'
The API returns a presigned GET URL in result.dest_url
where the file from the Secure Share can be downloaded.
{
"request_id": "prq_7dmcm64tyzxm7hlsbfhz33l577ifhyzm",
"status": "Success",
"summary": "Object get successful.",
"result": {
"dest_url": "https://pangea-share.s3.us-west-2.amazonaws.com/share-tmp/prq_7dmcm64tyzxm7hlsbfhz33l577ifhyzm?...",
"object": {
"folder": "/",
"id": "pos_pwwejxtwoaamkvjqy4gzscetchexoglp",
. . .
}
}
. . .
}
multipart
The multipart
transfer method is an option that does not leverage presigned URLs. It can be utilized for both providing an input file and receiving an output file, as outlined below:
-
When the API call requires an input file, the caller uploads the file encoded as
multipart/form-data
using the POST method as part of the initial request.To get the final result, make the following requests:
-
Initiate a service operation (such as File Scan).
POSTmultipart/form-datacURLcurl --location 'https://file-scan.aws.us.pangea.cloud/v1/scan' \ --header "Authorization: Bearer $PANGEA_FILE_SCAN_TOKEN" \ --form 'request="{\"transfer_method\":\"multipart\"}";type=application/json' \ --form "file=@$PATH_TO_FILE"
response/with/results/locationjson{ "request_id": "prq_3f2hlqm3nkd23tuqpfcsxymhkskyvlgp", "result": { "location": "https://file-scan.aws.us.pangea.cloud/request/prq_3f2hlqm3nkd23tuqpfcsxymhkskyvlgp", . . . }, "status": "Accepted", "summary": "Your request is in progress. Use 'result, location' below to poll for results. See https://pangea.cloud/docs/api/async?service=file-scan&request_id=prq_3f2hlqm3nkd23tuqpfcsxymhkskyvlgp for more information." }
-
Check the results of the requested operation.
The file to be processed is uploaded as a part of the initial request, and you can immediately start checking the results of the requested service operation using the
result.location
URL returned in the response.GETresults/of/service/requestcURLcurl --location 'https://file-scan.aws.us.pangea.cloud/request/prq_3f2hlqm3nkd23tuqpfcsxymhkskyvlgp' \ --header "Authorization: Bearer $PANGEA_FILE_SCAN_TOKEN"
results/of/service/requestjson{ "request_id": "prq_lnzw6zpns2z6e4xyyzrb26ssjod7hh7o", "status": "Success", "summary": "File was scanned", "result": { "data": { "verdict": "benign", . . . } }, . . . }
-
-
In cases where the API call generates an output file and the
transfer_method
ismultipart
, the 200 response will contain the file encoded as amultipart/form-data
octet-stream
.When requesting a download from the Secure Share using its /get or /get_archive API endpoints, you have the option to include the file content in the response. This can be done by selecting the
multipart
transfer method option.In your request, include the following parameters:
- "transfer_method": "multipart"
- "id" or "path": Provide either the ID or path to a file or folder within the Secure Share.
Send the request data in JSON format:
POSTrequest/file/with/dest-urlcURLcurl --location 'https://share.aws.us.pangea.cloud/v1/get' \ --header 'Content-Type: application/json' \ --header "Authorization: Bearer $PANGEA_SECURE_SHARE_TOKEN" \ --data '{ "transfer_method": "multipart", "id": "pos_bbydypfioc2vta4zf7jdryw6a6j3p5ne" }' --remote-name --remote-header-name
The file content is included in the response and can be saved in the local file system.
share-id
The Sanitize service can be used against a file saved in the Secure Share. To make the API call, you need to specify the following parameters in your initial request:
- "transfer_method": "share-id"
- "share_id": The ID of the file saved in the Secure Share.
A successful response from the Sanitize service will contain a presigned GET URL in result.dest_url
, which you can use to download the sanitized output.
To get the final result, make the following requests:
-
Request sanitization.
Send the request data in JSON format:
POSTrequest/sanitize/with/share-idcURLcurl --location 'https://sanitize.aws.us.pangea.cloud/v1/sanitize' \ --header 'Content-Type: application/json' \ --header "Authorization: Bearer $PANGEA_SANITIZE_TOKEN" \ --data '{ "transfer_method": "share-id", "share_id": "pos_jxap2xe7guejee3udi5bc4mu4tbdol5w" }'
After making a call to the Sanitize service, you'll receive an Asynchronous Response. This response contains a GET URL in
result.location
, which you can use to track the status of your request.request/sanitize/with/share-idjson{ "request_id": "prq_xao2vkxsxbk37fgf5zmdwh3vgiterc34", "status": "Accepted", "summary": "Your request is in progress. Use 'result, location' below to poll for results. See https://pangea.cloud/docs/api/async?service=sanitize&request_id=prq_xao2vkxsxbk37fgf5zmdwh3vgiterc34 for more information.", "result": { "location": "https://sanitize.aws.us.pangea.cloud/request/prq_xao2vkxsxbk37fgf5zmdwh3vgiterc34", . . . }, . . . }
-
Check the results of the requested sanitization.
GETresults/of/sanitizecURLcurl --location 'https://sanitize.aws.us.pangea.cloud/request/prq_xao2vkxsxbk37fgf5zmdwh3vgiterc34' \ --header "Authorization: Bearer $PANGEA_SANITIZE_TOKEN"
Use the presigned GET URL in
result.dest_url
within the response to download the sanitized output.results/of/sanitizejson{ "request_id": "prq_xao2vkxsxbk37fgf5zmdwh3vgiterc34", "result": { "dest_url": "https://pangea-sanitize-input.s3.us-west-2.amazonaws.com/2024030423/prq_64tjspdh4yxxownpm2ap4qb4rbxuedeo/sanitized.my_tiny.pdf?...", . . . }, "status": "Success", "summary": "Successfully completed the request. The file download link is valid for 24h0m0s." }
Additional Parameters for the Sanitize Service
share_output
You can choose how the results of a Sanitize API call are delivered by optionally specifying an additional parameter, share_output
.
For a Sanitize API service call, an input file can be provided using different transfer_method
options discussed earlier:
- "source-url"
- "put-url"
- "post-url"
- "share-id"
- "multipart"
You have two options for receiving the results of a Sanitize API call:
-
If you omit the optional
share_output
parameter in your initial request, the successful response from the Sanitize service will contain a presigned GET URL inresult.dest_url
, which you can use to download the sanitized output.For example:
-
Request sanitization.
POSTsanitize/file/at/source-urlcURLcurl --location 'https://sanitize.aws.us.pangea.cloud/v1/sanitize' \ --header 'Content-Type: application/json' \ --header "Authorization: Bearer $PANGEA_SANITIZE_TOKEN" \ --data '{ "transfer_method": "source-url", "source_url": "https://my-scan-input.s3.us-west-2.amazonaws.com/samples/my_tiny.pdf?..." }'
After making a call to the Sanitize service, you'll receive an Asynchronous Response. This response contains a GET URL in
result.location
, which you can use to track the status of your request.response/with/results/locationjson{ "request_id": "prq_64tjspdh4yxxownpm2ap4qb4rbxuedeo", "status": "Accepted", "summary": "Your request is in progress. Use 'result, location' below to poll for results. See https://pangea.cloud/docs/api/async?service=sanitize&request_id=prq_64tjspdh4yxxownpm2ap4qb4rbxuedeo for more information.", "result": { "location": "https://sanitize.aws.us.pangea.cloud/request/prq_64tjspdh4yxxownpm2ap4qb4rbxuedeo", . . . }, . . . }
-
Check the results of the requested sanitization.
GETresults/of/sanitizecURLcurl --location 'https://sanitize.aws.us.pangea.cloud/request/prq_64tjspdh4yxxownpm2ap4qb4rbxuedeo' \ --header "Authorization: Bearer $PANGEA_SANITIZE_TOKEN"
Use the presigned GET URL in
result.dest_url
to download the sanitized output.results/of/sanitizejson{ "request_id": "prq_64tjspdh4yxxownpm2ap4qb4rbxuedeo", "result": { "dest_url": "https://pangea-sanitize-input.s3.us-west-2.amazonaws.com/2024030423/prq_64tjspdh4yxxownpm2ap4qb4rbxuedeo/sanitized.my_tiny.pdf?...", . . . }, "status": "Success", "summary": "Successfully completed the request. The file download link is valid for 24h0m0s." }
-
-
Enabling
share_output
in your initial request saves the sanitized output in the Secure Share.For example:
-
Request sanitization.
POSTsanitize/file/at/source-urlcURLcurl --location 'https://sanitize.aws.us.pangea.cloud/v1/sanitize' \ --header 'Content-Type: application/json' \ --header "Authorization: Bearer $PANGEA_SANITIZE_TOKEN" \ --data '{ "transfer_method": "source-url", "source_url": "https://pangea-sanitize-input.s3.us-west-2.amazonaws.com/samples/redact_tiny.pdf?...", "share_output": { "enabled": true, "output_folder": "/" } }'
Note that if you specify a non-existent "output_folder" location, the Secure Share will automatically create it for you.
In the response, you'll receive a GET URL in the
result.location
. You can use this URL to check the status of the call and its eventual results.response/with/results/locationjson{ "request_id": "prq_zrdj2aggcspg6nslzlk7im63s577o34z", "result": { "location": "https://sanitize.aws.us.pangea.cloud/request/prq_zrdj2aggcspg6nslzlk7im63s577o34z", . . . }, "status": "Accepted", "summary": "Your request is in progress. Use 'result, location' below to poll for results. See https://pangea.cloud/docs/api/async?service=sanitize&request_id=prq_zrdj2aggcspg6nslzlk7im63s577o34z for more information.", . . . }
-
Check the results of the requested sanitization.
GETresults/of/sanitizecURLcurl --location 'https://sanitize.aws.us.pangea.cloud/request/prq_zrdj2aggcspg6nslzlk7im63s577o34z' \ --header "Authorization: Bearer $PANGEA_SANITIZE_TOKEN"
If the call is successful,
result.dest_share_id
will contain the ID of the file saved in the Secure Share.results/of/sanitizejson{ "request_id": "prq_zrdj2aggcspg6nslzlk7im63s577o34z", "status": "Success", "summary": "Successfully completed the request. The Sanitized file sanitized.Asynchronous API Responses Pangea.pdf can be found in the Secure Share under folder: /.", "result": { "dest_share_id": "pos_pp2l24fj7kcdafmyqtztd6oeoofpmeid", . . . }, . . . }
-
Example: Using transfer method post-url with the File Scan service
The Python code snippet below uses the post-url
transfer_method
and does the following:
-
Calls the
file_scan_endpoint
(File Scan API) withtransfer_method
ofpost-url
.-
The
crc32c
(hex-encoded),sha256
(hex-encoded), andsize
would need to be calculated from the file to be uploaded and set in these parameters (not shown). You can use the Presigned URL Helper to calculate these values. -
You also need to supply your File Scan authorization token represented by
SCAN_AUTH_TOKEN
in the code snippet below.
-
-
Extracts
final_scan_result_url
containing therequest_id
, thepost_url
, and thepost_form_data
from thefile_scan_response
. -
The
post_form_data
are multipart/form-data encoded as fields into the body of thePOST
request to thepost_url
. -
The contents of the file to be scanned are multipart/form-data encoded into the body as the final field of the
POST
request to thepost_url
. -
The file is sent as a
POST
to thepost_url
for File Scan to receive and scan. -
The
final_scan_result_url
is used to poll for a final File Scan result.
The following code snippet is only intended to be used as an illustration and does not contain all of the necessary code to properly make Presigned URL API calls.
import requests
# (SEE 1 ABOVE)
# Prepare the body of the initial POST
request_body = {
"verbose": true,
"raw": true,
"provider": "crowdstrike",
"transfer_method": "post-url",
"crc32c": "hex encoded crc_value_here",
"sha256": "hex encoded sha_value_here",
"size": size_value_here
}
# (SEE 1 ABOVE)
# Pangea authorization
headers = {
"Authorization": "Bearer " + SCAN_AUTH_TOKEN,
}
# (SEE 1 ABOVE)
# POST to the Pangea File Scan API endpoint
file_scan_endpoint = "https://file-scan.aws.us.pangea.cloud/v1/scan"
file_scan_response = requests.Request("POST", file_scan_endpoint, headers=headers, json=request_body)
session = requests.session()
session.send(file_scan_response)
# (SEE 2 ABOVE)
# Get the response and the values needed to post to the upload URL
status_code = file_scan_response.status_code
json_response = file_scan_response.json()
final_scan_result_url = json_response["result"]["location"]
post_form_data = json_response["result"]["post_form_data"]
post_url = json_response["result"]["post_url"]
# (SEE 3 ABOVE)
# Create a form-data encoded body for the upload that contains the post_form_data from the API call
body = []
for k, v in post_form_data.items():
body.append((k, v))
# (SEE 4 ABOVE)
# Add the file to be scanned to the POST
files = {
"file": ("filename.ext", file.open("<path to the file>"), "application/octet-stream"),
}
# (SEE 5 ABOVE)
# Upload the file to the post_url for File Scan to scan
post_url_upload_response = requests.post(post_url, data=body, files=files, verify=verify)
# (SEE 6 ABOVE)
# Poll for the File Scan result using the final_scan_result_url
while True:
result_response = requests.get(final_scan_result_url, headers=headers, verify=verify)
if result_response.status_code == 200:
result_data = result_response.json()
if result_data["status"] == "Completed":
print("Scan completed successfully")
# Process results if needed
break
elif result_data["status"] == "InProgress":
print("Scan still in progress. Polling...")
time.sleep(10) # Sleep for a while before polling again
else:
print("Scan status: " + result_data["status"])
break
else:
print("Error: Unable to retrieve scan result")
break
Presigned URL Helper
Upload a file to this helper to get the following data:
- The SHA-256 hash of the file.
- The CRC32C hash of the file.
- The size of the file (in bytes).