Skip to main content

Unredact Log Events

Learn how to recover encrypted audit log values that have been redacted using Format Preserving Encryption (FPE)

You can integrate Secure Audit Log with the Redact service to redact log events. When you enable FPE Redaction in the Redact configuration used in Secure Audit Log, logs redacted with FPE can be decrypted to their original values.

Unredact individual log events​

You can use the Redact /v1/unredact endpoint to decrypt data redacted with FPE.

Use the same Redact configuration that was applied when redacting the data. Copy the Default Token, Config ID, and Domain values from the Redact Overview page in the Pangea User Console, and use them in your call to the /v1/unredact endpoint.

export PANGEA_DOMAIN="aws.us.pangea.cloud"
export PANGEA_REDACT_TOKEN="pts_s2ngg2...hzwafm"
export PANGEA_REDACT_CONGIG_ID="pci_4ku3oviu6bpjsghhpch5hw2l564myecx"

Make a request to the /v1/unredact endpoint in your Pangea project domain. Use the Redact service token to authorize the request.

Provide the following parameters:

  • redacted_data - The redacted text with encrypted values.

    When using the Redact APIs directly, the redacted text is returned under result.redacted_text in the response from the /v1/redact endpoint.

  • fpe_context - The value needed to decrypt data redacted with FPE.

    To decrypt the original values redacted with FPE, you need the context in which the original text was redacted.

    When you use the Redact /v1/redact endpoint APIs directly, the entire context is returned as an opaque value under result.fpe_context in the response.

  • config_id - The Redact service configuration ID.

    The Redact service can have multiple configurations.

    If you use the same token for multiple service configurations, the token alone is not sufficient to determine which configuration you are requesting. In this case, you MUST specify a configuration ID when calling the service APIs. Otherwise, you will receive an AmbiguousConfigID error.

The fpe_context required to unredact log event data is attached to each audit record redacted with FPE and is specific to that record. You can get the fpe_context value from the results returned from the Secure Audit Service /v1/search API endpoint by setting the return_context parameter to true.

Unredact multiple log events​

You can use the Redact /v1/unredact_events endpoint to decrypt data redacted with FPE in an array of log events.

Make a request to the /v1/unredact_events endpoint in your Pangea project domain. Use the Redact service token to authorize the request.

Provide the following parameters:

Example​

Enable FPE redaction in Secure Audit Log​

Enable FPE redaction and select the fields you want to redact with FPE, as described in the Redact Records documentation. For example, enable FPE redaction on the actor field:

Enable FPE redaction on the actor field in Secure Audit Log Redact Records page in the Pangea User Console
Enable FPE redaction on actor

Enable the Format Preserving Encryption (FPE) redaction method for the Email Address (EMAIL_ADDRESS) and Location (LOCATION) rules, as described in the Redact Rulesets configuration documentation:

Select the FPE redaction method for the Email Address and Location rules on the Redact Rulesets page in the Pangea User Console

Select the FPE redaction method for Email Address and Location

Log events​

Specify the Secure Audit Log and Redact parameters for redacting log events:

export PANGEA_DOMAIN="aws.us.pangea.cloud"
export PANGEA_AUDIT_TOKEN="pts_v3cb4s...u4jnej"
export PANGEA_AUDIT_CONFIG_ID="pci_k4scvz7mkp2o6pz5fwfx5fihdv4rxoi6"
export PANGEA_REDACT_TOKEN="pts_s2ngg2...hzwafm"
export PANGEA_REDACT_CONFIG_ID="pci_vm7kmflvlfk5dxngfbvhxp5cokw4ona7"

You can log a single event using the /v1/log endpoint:

POST: /v1/log
curl --location "https://audit.$PANGEA_DOMAIN/v1/log" \
--header 'Content-Type: application/json' \
--header "Authorization: Bearer $PANGEA_AUDIT_TOKEN" \
--data-raw '{
  "config_id": "'"$PANGEA_AUDIT_CONFIG_ID"'",
  "event": {
    "message": "Security for the containment units on Isla Nublar, Costa Rica, has been updated.",
    "actor": "dennis.nedry@ingens.com",
    "action": "Update",
    "new": "off",
    "old": "on"
  }
}'
/v1/log response
{
  "status": "Success",
  "summary": "Logged 1 record(s)",
  "result": {
    "hash": "a82fefc842412c4e8d3b6c912e1a002e29588768eccc7505be860e6e61eb0982"
  },
  ...
}

Alternatively, you can log multiple events in bulk using the /v2/log endpoint:

POST: /v2/log
curl --location "https://audit.$PANGEA_DOMAIN/v2/log" \
--header 'Content-Type: application/json' \
--header "Authorization: Bearer $PANGEA_AUDIT_TOKEN" \
--data-raw '{
  "config_id": "pci_k4scvz7mkp2o6pz5fwfx5fihdv4rxoi6",
  "events": [
    {
      "event": {
        "message": "Phone lines to the Palo Alto office have been put on hold.",
        "actor": "dennis.nedry@ingens.com",
        "action": "Block",
        "new": "busy",
        "old": "free"
      }
    },
    {
      "event": {
        "message": "Login attempts on Isla Nublar, Costa Rica, exceeded the allowable limit.",
        "actor": "dennis.nedry@ingens.com",
        "action": "Lock",
        "new": "locked",
        "old": "unlocked"
      }
    }
  ]
}'
/2/log response
{
  "status": "Success",
  "summary": "Logged 2 record(s)",
  "result": {
    "results": [
      {
        "hash": "57fee37ad301aa0dfa237d3f3a0a718bfabacf61bcac8f2aaec679c3d9eed64e"
      },
      {
        "hash": "99a18388ef29a7e427226bc64737325d8dd510ed8794c5c42b39d6649f0945aa"
      }
    ]
  }
}

Search for audit records​

Use the Secure Audit Log /v1/search endpoint to search for audit records. Remember to include the return_context parameter populated with true to include the FPE context in the response:

POST: /v1/search
curl --location "https://audit.$PANGEA_DOMAIN/v1/search" \
--header 'Content-Type: application/json' \
--header "Authorization: Bearer $PANGEA_AUDIT_TOKEN" \
--data '{
  "config_id": "'"$PANGEA_AUDIT_CONFIG_ID"'",
  "return_context": true
}'
note

The response from the /v1/search endpoint might be asynchronous and return a location for polling results:

/v1/search async response
{
  "status": "Accepted",
  "summary": "Your request is in progress. Use 'result, location' below to poll for results. See https://dev.pangea.cloud/docs/api/async?service=audit&request_id=prq_2qngfew2zd54nwxsswkbxstjshkqhvso for more information.",
  "result": {
    "location": "https://audit.us.dev.pangea.cloud/request/prq_2qngfew2zd54nwxsswkbxstjshkqhvso",
    ...
  },
  ...
}

Poll the URL returned in result.location until the response status changes from "Accepted" to "Success" or until a failure is reported:

GET: /v1/search
curl --location "<result.location>" \
--header "Authorization: Bearer $PANGEA_AUDIT_TOKEN"

When successful, the response will include the search results. Note the encrypted content in the result.events[0].envelop.event.actor value and the result.events[0].fpe_context value associated with this event:

/v1/search response
{
  "status": "Success",
  "summary": "Found 3 event(s)",
  "result": {
    "id": "pas_75yhsbdx4wpigl2ymd44tawwfvqusgyx",
    "count": 3,
    "expires_at": "2024-09-13T15:59:15.790546Z",
    "events": [
      {
        "envelope": {
          "received_at": "2024-09-11T16:07:06.851648Z",
          "event": {
            "message": "Login attempts on afBd FYP6RN, CjzoH 9I9e, exceeded the allowable limit.",
            "actor": "4HJalH.004zg@MVmnx6.iuc",
            "action": "Lock",
            "new": "locked",
            "old": "unlocked"
          }
        },
        "hash": "89f6b9798b1e246a00b4bee2c587d6407d36149b4fcaf163b75de75e989b4f2c",
        "fpe_context": "eyJhIjogIkFFUy1GRjMtMS0yNTYtQkVUQSIsICJ0IjogIjRFSWFuZ08iLCAibSI6IFt7ImEiOiAzLCAicyI6IDAsICJlIjogMjMsICJrIjogImFjdG9yIn0sIHsiYSI6IDMsICJzIjogMTgsICJlIjogMjksICJrIjogIm1lc3NhZ2UifSwgeyJhIjogMywgInMiOiAzMSwgImUiOiA0MSwgImsiOiAibWVzc2FnZSJ9XSwgImsiOiAicHZpX3FxcTNndXRocm14ZXk1c3BuZ3RzZ3N2cGt6NWFqbHZhIiwgInYiOiAxLCAiYyI6ICJwY2lfaXJzM2Jva2p1dGpyYXN0aXc0azNrN3RqemY2dDYydHEifQ==",
        "published": false,
        "membership_proof": "l:6c872888b343121dc306710220c272a147eeb74c40982e50a14f14bc0f62c766,l:440bc16cc4833203d02a40582938e26e5a86b13d4c5bbe2e325147caf1ac0bd0"
      },
      {
        "envelope": {
          "received_at": "2024-09-11T16:07:06.851648Z",
          "event": {
            "message": "Phone lines to the lwYx CQFg office have been put on hold.",
            "actor": "4HJalH.004zg@MVmnx6.iuc",
            "action": "Block",
            "new": "busy",
            "old": "free"
          }
        },
        "hash": "d1f9617571e7273204875b4ac25f32b2173aa6cd4fd38781606521b082905fb7",
        "fpe_context": "eyJhIjogIkFFUy1GRjMtMS0yNTYtQkVUQSIsICJ0IjogIjRFSWFuZ08iLCAibSI6IFt7ImEiOiAzLCAicyI6IDAsICJlIjogMjMsICJrIjogImFjdG9yIn0sIHsiYSI6IDMsICJzIjogMTUsICJlIjogMjQsICJrIjogIm1lc3NhZ2UifV0sICJrIjogInB2aV9xcXEzZ3V0aHJteGV5NXNwbmd0c2dzdnBrejVhamx2YSIsICJ2IjogMSwgImMiOiAicGNpX2lyczNib2tqdXRqcmFzdGl3NGszazd0anpmNnQ2MnRxIn0=",
        "published": false,
        "membership_proof": "l:29c4e3ed93712f753f820cb428deaac5d37c9ef4cb5055688464725125939b13,r:89f6b9798b1e246a00b4bee2c587d6407d36149b4fcaf163b75de75e989b4f2c,l:440bc16cc4833203d02a40582938e26e5a86b13d4c5bbe2e325147caf1ac0bd0"
      },
      {
        "envelope": {
          "received_at": "2024-09-11T16:07:03.452619Z",
          "event": {
            "message": "Security for the containment units on ZjTE LQZwWN, eTVYt qqJb, has been updated.",
            "actor": "2KxLDR.tNzvH@Ug6OsB.P70",
            "action": "Update",
            "new": "off",
            "old": "on"
          }
        },
        "hash": "29c4e3ed93712f753f820cb428deaac5d37c9ef4cb5055688464725125939b13",
        "fpe_context": "eyJhIjogIkFFUy1GRjMtMS0yNTYtQkVUQSIsICJ0IjogIkx5MmxLUmoiLCAibSI6IFt7ImEiOiAzLCAicyI6IDAsICJlIjogMjMsICJrIjogImFjdG9yIn0sIHsiYSI6IDMsICJzIjogMzgsICJlIjogNDksICJrIjogIm1lc3NhZ2UifSwgeyJhIjogMywgInMiOiA1MSwgImUiOiA2MSwgImsiOiAibWVzc2FnZSJ9XSwgImsiOiAicHZpX3FxcTNndXRocm14ZXk1c3BuZ3RzZ3N2cGt6NWFqbHZhIiwgInYiOiAxLCAiYyI6ICJwY2lfaXJzM2Jva2p1dGpyYXN0aXc0azNrN3RqemY2dDYydHEifQ==",
        "published": false,
        "membership_proof": "r:d1f9617571e7273204875b4ac25f32b2173aa6cd4fd38781606521b082905fb7,r:89f6b9798b1e246a00b4bee2c587d6407d36149b4fcaf163b75de75e989b4f2c,l:440bc16cc4833203d02a40582938e26e5a86b13d4c5bbe2e325147caf1ac0bd0"
      }
    ],
    ...
  }
}

Unredact a single audit record​

Use the Redact /v1/unredact endpoint to decrypt data redacted with FPE in a single log event.

In your API call, provide result.events[<index>].fpe_context and the stringified content of result.events[<index>].envelope.event from the event data in the response from /v1/search. For example:

POST: /v1/unredact
curl --location "https://redact.$PANGEA_DOMAIN/v1/unredact" \
--header 'Content-Type: application/json' \
--header "Authorization: Bearer $PANGEA_REDACT_TOKEN" \
--data-raw '{
  "config_id": "pci_vm7kmflvlfk5dxngfbvhxp5cokw4ona7",
  "redacted_data": {"message":"Security for the containment units on ZjTE LQZwWN, eTVYt qqJb, has been updated.","actor":"2KxLDR.tNzvH@Ug6OsB.P70","action":"Update","new":"off","old":"on"},
  "fpe_context": "eyJhIjogIkFFUy1GRjMtMS0yNTYtQkVUQSIsICJ0IjogIkx5MmxLUmoiLCAibSI6IFt7ImEiOiAzLCAicyI6IDAsICJlIjogMjMsICJrIjogImFjdG9yIn0sIHsiYSI6IDMsICJzIjogMzgsICJlIjogNDksICJrIjogIm1lc3NhZ2UifSwgeyJhIjogMywgInMiOiA1MSwgImUiOiA2MSwgImsiOiAibWVzc2FnZSJ9XSwgImsiOiAicHZpX3FxcTNndXRocm14ZXk1c3BuZ3RzZ3N2cGt6NWFqbHZhIiwgInYiOiAxLCAiYyI6ICJwY2lfaXJzM2Jva2p1dGpyYXN0aXc0azNrN3RqemY2dDYydHEifQ=="
}'

The response from the /v1/unredact endpoint will include the count of unredacted entries and the original data with decrypted values where the FPE redaction was applied under the result.data key:

/v1/unredact response
{
  "status": "Success",
  "summary": "Success. Unredacted 3 item(s) from items",
  "result": {
    "data": {
      "message": "Security for the containment units on Isla Nublar, Costa Rica, has been updated.",
      "actor": "dennis.nedry@ingens.com",
      "action": "Update",
      "new": "off",
      "old": "on"
    }
  },
  ...
}

Unredact multiple audit records​

Use the Redact /v1/unredact_events endpoint to decrypt data redacted with FPE in multiple log events.

In your API call, provide result.events and the stringified content of result.events from the event data in the response from /v1/search, or a subset of this array. For example:

POST: /v1/unredact_events
curl --location "https://redact.$PANGEA_DOMAIN/v1/unredact_events" \
--header 'Content-Type: application/json' \
--header "Authorization: Bearer $PANGEA_REDACT_TOKEN" \
--data-raw '{
  "config_id": "pci_vm7kmflvlfk5dxngfbvhxp5cokw4ona7",
  "events": [{"envelope":{"received_at":"2024-09-11T16:07:06.851648Z","event":{"message":"Login attempts on afBd FYP6RN, CjzoH 9I9e, exceeded the allowable limit.","actor":"4HJalH.004zg@MVmnx6.iuc","action":"Lock","new":"locked","old":"unlocked"}},"hash":"89f6b9798b1e246a00b4bee2c587d6407d36149b4fcaf163b75de75e989b4f2c","fpe_context":"eyJhIjogIkFFUy1GRjMtMS0yNTYtQkVUQSIsICJ0IjogIjRFSWFuZ08iLCAibSI6IFt7ImEiOiAzLCAicyI6IDAsICJlIjogMjMsICJrIjogImFjdG9yIn0sIHsiYSI6IDMsICJzIjogMTgsICJlIjogMjksICJrIjogIm1lc3NhZ2UifSwgeyJhIjogMywgInMiOiAzMSwgImUiOiA0MSwgImsiOiAibWVzc2FnZSJ9XSwgImsiOiAicHZpX3FxcTNndXRocm14ZXk1c3BuZ3RzZ3N2cGt6NWFqbHZhIiwgInYiOiAxLCAiYyI6ICJwY2lfaXJzM2Jva2p1dGpyYXN0aXc0azNrN3RqemY2dDYydHEifQ==","published":true,"leaf_index":192,"membership_proof":"l:6c872888b343121dc306710220c272a147eeb74c40982e50a14f14bc0f62c766,l:440bc16cc4833203d02a40582938e26e5a86b13d4c5bbe2e325147caf1ac0bd0,l:251ca87214dcf18171295df41c427154f6c9af4311d4a38bf55a8714de550fdc,l:b11c63b63329b2b0fd8dbc5d25c8ba3785926952b066b4f8bf44c06122f70e2e"},{"envelope":{"received_at":"2024-09-11T16:07:06.851648Z","event":{"message":"Phone lines to the lwYx CQFg office have been put on hold.","actor":"4HJalH.004zg@MVmnx6.iuc","action":"Block","new":"busy","old":"free"}},"hash":"d1f9617571e7273204875b4ac25f32b2173aa6cd4fd38781606521b082905fb7","fpe_context":"eyJhIjogIkFFUy1GRjMtMS0yNTYtQkVUQSIsICJ0IjogIjRFSWFuZ08iLCAibSI6IFt7ImEiOiAzLCAicyI6IDAsICJlIjogMjMsICJrIjogImFjdG9yIn0sIHsiYSI6IDMsICJzIjogMTUsICJlIjogMjQsICJrIjogIm1lc3NhZ2UifV0sICJrIjogInB2aV9xcXEzZ3V0aHJteGV5NXNwbmd0c2dzdnBrejVhamx2YSIsICJ2IjogMSwgImMiOiAicGNpX2lyczNib2tqdXRqcmFzdGl3NGszazd0anpmNnQ2MnRxIn0=","published":true,"leaf_index":192,"membership_proof":"l:29c4e3ed93712f753f820cb428deaac5d37c9ef4cb5055688464725125939b13,r:89f6b9798b1e246a00b4bee2c587d6407d36149b4fcaf163b75de75e989b4f2c,l:440bc16cc4833203d02a40582938e26e5a86b13d4c5bbe2e325147caf1ac0bd0,l:251ca87214dcf18171295df41c427154f6c9af4311d4a38bf55a8714de550fdc,l:b11c63b63329b2b0fd8dbc5d25c8ba3785926952b066b4f8bf44c06122f70e2e"},{"envelope":{"received_at":"2024-09-11T16:07:03.452619Z","event":{"message":"Security for the containment units on ZjTE LQZwWN, eTVYt qqJb, has been updated.","actor":"2KxLDR.tNzvH@Ug6OsB.P70","action":"Update","new":"off","old":"on"}},"hash":"29c4e3ed93712f753f820cb428deaac5d37c9ef4cb5055688464725125939b13","fpe_context":"eyJhIjogIkFFUy1GRjMtMS0yNTYtQkVUQSIsICJ0IjogIkx5MmxLUmoiLCAibSI6IFt7ImEiOiAzLCAicyI6IDAsICJlIjogMjMsICJrIjogImFjdG9yIn0sIHsiYSI6IDMsICJzIjogMzgsICJlIjogNDksICJrIjogIm1lc3NhZ2UifSwgeyJhIjogMywgInMiOiA1MSwgImUiOiA2MSwgImsiOiAibWVzc2FnZSJ9XSwgImsiOiAicHZpX3FxcTNndXRocm14ZXk1c3BuZ3RzZ3N2cGt6NWFqbHZhIiwgInYiOiAxLCAiYyI6ICJwY2lfaXJzM2Jva2p1dGpyYXN0aXc0azNrN3RqemY2dDYydHEifQ==","published":true,"leaf_index":192,"membership_proof":"r:d1f9617571e7273204875b4ac25f32b2173aa6cd4fd38781606521b082905fb7,r:89f6b9798b1e246a00b4bee2c587d6407d36149b4fcaf163b75de75e989b4f2c,l:440bc16cc4833203d02a40582938e26e5a86b13d4c5bbe2e325147caf1ac0bd0,l:251ca87214dcf18171295df41c427154f6c9af4311d4a38bf55a8714de550fdc,l:b11c63b63329b2b0fd8dbc5d25c8ba3785926952b066b4f8bf44c06122f70e2e"}]
}'
note

Each redacted log event has its own FPE context.

The response from the /v1/unredact_events endpoint will include the count of unredacted events and the original data with decrypted values where the FPE redaction was applied under the result.events key:

/v1/unredact response
{
  "status": "Success",
  "summary": "Success. Unredacted 3 item(s) from items",
  "result": {
    "events": [
      {
        "message": "Login attempts on Isla Nublar, Costa Rica, exceeded the allowable limit.",
        "actor": "dennis.nedry@ingens.com",
        "action": "Lock",
        "new": "locked",
        "old": "unlocked"
      },
      {
        "message": "Phone lines to the Palo Alto office have been put on hold.",
        "actor": "dennis.nedry@ingens.com",
        "action": "Block",
        "new": "busy",
        "old": "free"
      },
      {
        "message": "Security for the containment units on Isla Nublar, Costa Rica, has been updated.",
        "actor": "dennis.nedry@ingens.com",
        "action": "Update",
        "new": "off",
        "old": "on"
      }
    ]
  },
  ...
}

Was this article helpful?

Contact us