Skip to main content

Secure Audit Log SMS Proxy

Learn how to create a cryptographically verifiable audit trail of Twilio-powered SMS conversations using the Secure Audit Log service and redact sensitive information in real time with the Redact Service. By publishing conversation hashes to the blockchain you can provide irrefutable evidence of what was said between two parties without actually exposing the contents of the conversation publicly. With Pangea’s Secure Audit Log SDK, you can deliver the power of the blockchain to your users with a few lines of code. In this tutorial, you’ll build an SMS forwarding service that logs messages between two parties and optionally configure it to redact sensitive or personally identifiable information (PII) in real-time with Pangea's Redact Rulesets.

Requirements​

Get the sample code​

Step 1: Clone the Pangea Builders sample repo from GitHub:

git clone https://github.com/pangeacyber/audit-sms-proxy.git

Step 2: Change the working directory to the Django project with the following command:

cd audit-sms-proxy/python

Take a moment to explore the project files and configure it to your environments.

  • manage.py - Django's command-line utility for administrative tasks, such as running a development web server
  • .env - Contains the environment variables the app will reference.
  • audit/views.js - The application source file that contains a single function to handle incoming SMS messages.

Step 3: Install the Python modules used by the application:

From the root of the project directory run each of the following commands to install each dependency.

To install the Pangea SDK:

pip3 install pangea-sdk

To install the Django web framework:

pip3 install Django

To install the Twilio SDK:

pip3 install twilio

To install the dotenv module you’ll use to load the .env file into environment variables:

pip3 install python-dotenv

Configure and deploy the app​

The application source reads 6 variables from the .env file.

  • ACCOUNT_SID and AUTH_TOKEN to authenticate your app with the Twilio service.
  • PANGEA_DOMAIN and PANGEA_AUTH_TOKEN to authenticate with the Pangea service.
  • TARGET_NUMBER and OWNER_NUMBER are used to determine where to forward incoming SMS messages.

Modify the .env file by replacing each {REPLACE} tag with the corresponding value. For the PANGEA_ specific variables, use the three values you noted in the Getting Started guide, or retrieve them from the Pangea Console by navigating to the Secure Audit Log tab. The Twilio values for ACCOUNT_SID and AUTH_TOKEN can be found on the landing page of the Twilio Console . OWNER_NUMBER and TARGET_NUMBER should each be a valid E.164 phone number you’d like to test with. For example, OWNER_NUMBER can be set to your mobile phone number and TARGET_NUMBER to a friend's number who you’d like to start and record an auditable message thread with. An example of an E.164 formatted number in the US is +16502223333.

NOTE: You can also set both OWNER_NUMBER and TARGET_NUMBER to your mobile number and reply to your own messages.

Verify your changes to the .env file with the git diff sub-command:

git diff

The output should look similar to this:

diff --git a/.env b/.env
index 7442c6a..9fd8304 100644
--- a/.env
+++ b/.env
@@ -1,6 +1,6 @@
-TWILIO_ACCOUNT_SID={REPLACE}
-TWILIO_AUTH_TOKEN={REPLACE}
-PANGEA_DOMAIN={REPLACE}
-PANGEA_AUTH_TOKEN={REPLACE}
-OWNER_NUMBER={REPLACE}
-TARGET_NUMBER={REPLACE}
+TWILIO_ACCOUNT_SID=ACXXXXXXXXXXXXXXXXXXXXXXXXXX
+TWILIO_AUTH_TOKEN=XXXXXXXXXXXXXXXXXXXXXXXXX
+PANGEA_DOMAIN=aws.us.pangea.cloud
+PANGEA_AUTH_TOKEN=pts_XXXXXXXXXXXXXXXXXXXXXXXX
+OWNER_NUMBER=+1305XXXXXXX
+TARGET_NUMBER=+1305XXXXXXX

Code walkthrough​

Inspect the contents of the /audit/views.py source file. The user-defined environment variables set in the .env file will be loaded by the load_dotenv() function.

# Load the .env file into environment variables
from dotenv import load_dotenv
load_dotenv()

Then, read each of the configured numbers into local variables.

# Read the target recipients numbers from environment variables
ownerNumber = os.getenv("OWNER_NUMBER")
targetNumber = os.getenv("TARGET_NUMBER")

The PANGEA_ values are assigned to their respective variables and the Pangea SDK classes are imported. A PangeaConfig object is created and used to create an Audit instance.

# Read the Pangea Domain and Auth Token from the environment variables
pangeaDomain = os.getenv("PANGEA_DOMAIN")
auditToken = os.getenv("PANGEA_AUTH_TOKEN")

# Import the Pangea SDK
from pangea.config import PangeaConfig
from pangea.services import Audit
from pangea.services.audit import Event
from pangea.services.audit import AuditException

# Instantiate a Pangea Configuration object with the end point domain
auditConfig = PangeaConfig(domain=pangeaDomain)
auditService = Audit(auditToken, config=auditConfig)

Similarly, the TWILIO_ account variables are read and set to accountSid and authToken, respectively, and used to represent an instance of the Client class defined in the Twilio SDK.

# Read the Twilio SID and Auth Token from the environment variables
accountSid = os.getenv("TWILIO_ACCOUNT_SID")
authToken = os.getenv("TWILIO_AUTH_TOKEN")

# Import the Twilio SDK
from twilio.rest import Client
from twilio.twiml.messaging_response import MessagingResponse

# Instantiate a Twilio Client using the accountSid and authToken
twilioClient = Client(accountSid, authToken)

The code reviewed thus far will execute when the Django app is loaded by the web server. Next, declare a function to handle requests. You will configure Twilio to invoke this function each time your Twilio-owned number receives an SMS. The function’s response and actions taken will be determined by the contents of the object passed in as the request parameter.

@require_POST
@csrf_exempt
def index(request):

    print(f"Event: {request.POST}")

    # Define a response object, in case a response to the sender is required
    resp = MessagingResponse()

Before invoking the Audit Service, determine if the inbound message should be added to the secure audit log and if so who to relay the message to after it is logged. Only messages between the two numbers configured in the .env file should be logged. Determine the destinationNumber to relay the message to.

  • If the message originated from the owner, send it to the target.
  • If the message came from the target, send it to the owner.
  • If the message came from any other number, notify the sender that the message will be ignored by populating and returning the MessageResponse.
# Determine the destination number
if request.POST['From'].endswith(ownerNumber):
  # If the message is from the owner, send it to target
  destinationNumber = targetNumber
elif request.POST['From'].endswith(targetNumber):
  # If the message is form the target, send it to owner
  destinationNumber = ownerNumber
else:
  # If the message is from any other number, reply to the sender
  resp.message("AUTOMATED RESPONSE: This is a private communication channel to securely record auditable conversations. Your message will be ignored!")
  return HttpResponse(resp)

Create an audit Event object and map the relevant details to the log entry. Set the actor field to the Twilio-owned number that is logging and forwarding the message. The source is set to the number that sent the incoming SMS, read from the requests From field, and target is set to the intended recipient or destinationNumber. The message is set to the body or contents of the SMS, also read from the incoming request.

# The number the original message was sent to is the number of the proxy
proxyNumber = request.POST['To'];

# The message originally sent to the poxy.
originalMessage = request.POST['Body']

# Map the Twilio event details to the Pangea Event object, for example, the
# source is set to the number that sent the message and the target is the
# recipient.
auditData = Event(
    actor=proxyNumber,
    source=request.POST['From'],
    target=destinationNumber,
    message=originalMessage,
    status=request.POST['SmsStatus'],
    action="forwarded",
)

Call the log method of the auditService with the auditData. This will invoke the Pangea Service to create the log entry. The hash of the message will also be recorded on a tamper-proof blockchain which can then be used to prove the conversation has not been modified. You can explore the log viewer on the Pangea Console to verify the message integrity later.

NOTE: The verbose parameter is set to true. This will instruct the Audit service to return a detailed response. You will need the details returned to determine if the logged message was modified by the Redact Rulesets.

try:
    auditResponse = auditService.log(event=auditData, verbose=True)
    print(f"Response: {auditResponse.result.dict(exclude_none=True)}")

The log function returns an auditResponse. If the response is marked successful, read the logged and potentially modified message from the response and use the twilioClient to relay the contents of the message to the destinationNumber.

if auditResponse.success:
    print(f"Forwarding message to: {destinationNumber}")

    loggedMessage = auditResponse.result.envelope.event.message

    # Send the logged message to the destinationNumber
    twilioResponse = twilioClient.messages.create(
        body=loggedMessage,
        from_=proxyNumber,
        to=destinationNumber
    )

    if twilioResponse.error_code is None:
        print("SMS successfully sent")

If the originalMessage received by the proxy is different than the loggedMessage returned by the Audit service, notify the sender that the message was modified before sending it to the recipient, by populating a message to the MessagingResponse represented by the twiml variable. Otherwise, the twilml will be returned with a blank response when the function completes.

  # If the logged message was modified by the redact rule set,
  # notify the sender via an automated response
  if loggedMessage != originalMessage:
    print("Redact detected")
    resp.message("AUTOMATED RESPONSE: You sent a message with sensitive, personal information. Our system redacted that information so that you can remain protected. The recipient of that message cannot access your sensitive information through this conversation.")

Run the app​

Django projects are bundled with the command-line utility, manage.py, to help you interact with your project. Use the following command to start a development server on you local machine:

python3 manage.py runserver 3000

You’ll see the following output on the command line:

Watching for file changes with StatReloader
Performing system checks...

System check identified no issues (0 silenced).

September 29, 2022 - 19:24:04
Django version 4.1.1, using settings 'audit_proxy.settings'
Starting development server at http://127.0.0.1:3000/
Quit the server with CONTROL-C

Your web app is now running! Your function is waiting to respond to requests sent to http://127.0.0.1:3000/audit/ but this URL is only reachable from your local computer. You’ll need to make the endpoint accessible publicly so Twilio can query it when an SMS is received.

Make your app accessible publicly on the internet with ngrok​

Twilio and other web services use the webhook callback pattern for event-driven interaction between two services on the Internet, in this case, Twilio and your web application. The ngrok program exposes services running locally on your machine via a url accessible from anywhere on the internet. Visit the link for details or follow these quick instructions to get started quickly:

On Linux and Mac:

./ngrok http 3000

Or on Windows:

ngrok http 3000

You should see the following output on the command line:

ngrok                                                                                                                                                     (Ctrl+C to quit)

Visit http://localhost:4040/ to inspect, replay, and modify your requests

Session Status                online
Account                       your name (Plan: Free)
Version                       3.1.0
Region                        United States (us)
Latency                       99ms
Web Interface                 http://127.0.0.1:4040
Forwarding                    https://4762-47-156-19-205.ngrok.io -> http://localhost:3000

Make a note of the Forwarding base URL above. It will be needed to configure your Twilio programmable number to query your app for instruction when an SMS is received.

Configure Twilio webhooks​

If you do not already have a Twilio number follow these instructions:

Screenshot

Otherwise, navigate to the Active numbers panel of the Twilio console and select the number you’d like to use with this service.

Screenshot

Under Messaging, look for the line that says A message comes in. Change the first box to Webhook and add the ngrok base URL with the path to the audit/ endpoint appended to it in the second box. Yours should look similar to https://[your generated ID].ngrok.io/audit. Save the configuration. The function on your local machine will now be invoked every time an SMS is sent to this number.

Screenshot

Congratulations! You just configured a Pangea-enabled Twilio service. You now have a secure audit trail between the OWNER_NUMBER and TARGET_NUMBER you configured in the .env file. Use a cellphone with either number to send an SMS to the Twilio number you configured to invoke your function. The SMS message will be logged by the Pangea service and forwarded to the other participant, similarly, their replies will be forwarded back to you creating a conversation thread on the SMS apps on both your phones.

Screenshot

To later view the conversation or present the verified proof that messages were not altered or deleted, navigate back to the Pangea Console , select Secure Audit Log from the left-hand navigation menu, and then select View Logs.

Screenshot

You can expand each message to view the sender and recipient details, labeled as source and target, respectively.

Screenshot

The green lock to the left of each message indicates that its hash has been published to the ARWeave blockchain and verified. To view the transaction on ViewBlock.io , click the lock icon and then click the View button next to Published Root.

NOTE: Publishing occurs once per hour so it may take up to an hour for a record to show as Verified.

Screenshot

Configure Redact rules​

You can configure the Audit service to redact specific data formats, such as emails, credit card numbers, and other sensitive or personal information. To do so, select Settings from the left-hand navigation menu, and then select Redact records.

Screenshot

If you have not yet enabled the Redact service, you will be presented with the Enable Redact Service option on the right-hand side. Click the Configure Redact button and review the benefits of the service. Then, click Next to continue.

Screenshot

Provide a new Token name, then select both Secure Audit Log and Redact from the Or choose from the list of individual services below section, and click Done.

Screenshot

From the Redact Overview dashboard, make a note of the TOKEN you just created and notice that both Redact and Audit are listed in the ACCESS RIGHTS column.

NOTE: You can quickly copy the token to your system's clipboard using the Screenshot shortcut.

Screenshot

Select Rulesets from the left-hand menu, to configure which types of data are redacted from log entries. Then, to redact email addresses and names, for example, select PII and enable the switches in the EMAIL_ADDRESS and PERSON sections. Then, click Save to update the rulesets.

Screenshot

Using the left-hand navigation menu, navigate back to Secure Audit Log > Setting. Select Redact records from the left column and enable the switch for Redact records. Then, click Save to update the settings change.

Screenshot

Update the token environment variable​

Your app must now provide the Pangea Audit SDK the new token with access rights to both the Audit and Redact service. To do so, update the PANGEA_AUTH_TOKEN value in the .env file and restart the app.

Restart the app​

  • Open the terminal window that is running the app and use CONTROL+C to terminate the web server.
  • When the command prompt reappears, use the following command to restart the server:
python3 manage.py runserver 3000

The app will now use the updated token to communicate with the Audit service.

Test the Redact rules​

Try sending a message that contains a name or email address, something like:

Hi, my name is Nicolas Vautier and my email is nicolas.vautier@pangea.cloud

You will receive an automated response each time a message is modified by the Redact service and the recipient will receive the modified message.

Screenshot

You can now update the Redact Settings from the Pangea Console at any time without redeploying the app. The redact rules will be applied to each new message sent through the proxy.

Conclusion​

In this article, you learned how to build and deploy an SMS proxy that records conversations and provides users with verifiable proof that the conversation was not altered. Most telecom providers maintain records of the messages sent over their network, but even the largest carriers are susceptible to tampered records on their centralized data sources. It may also be impractical for your users to request conversation logs from carriers as the process may require a legal process or subpoena. Utilizing a Blockchain with your Pangea-powered proxy you can quickly solve both these problems for your user.

Resources​