Policy Testing

You can use the AIDR Sandbox on the Prompt Rules tab to test rules directly where you define them.

To see how Access Rules and Prompt Rules work together, use the Playground feature on the Application collector details page. You can either use an existing Application collector registration or create a new one on the Collectors page.

Register Application collector

1. In the AIDR console, click + Collector >> Application >> Application >> Next.

2. In the Add a Collector dialog, enter a collector name and assign input and output policies.

   • Input Policy - Policy applied to data sent to the AI system.

     In the examples that follow, we refer to the input policy described in the Access Rules and Prompt Rules guides. You can also use any existing policy, such as Chat Input.

   • Output Policy - Policy applied to model responses, such as Chat Output.

   Leave the Async Report Only checkboxes unchecked for both policies.

   

3. Click Save to complete collector registration and open its details page.

4. On the collector details page, switch to the Playground tab.

Test Access Rules

With the Playground feature, you can do limited testing of access rules against the following request attributes:

• Application Name - Value specified in the Access Rules condition for the app.app_name attribute.
• Model - Value specified in the Access Rules condition for the model.model_name attribute.

For example, suppose the input policy selected in the top right has an access rule condition defined as:

Block

if (
  app.app_name == my-app
  or model.model_name == gpt-4o-mini
)

Given this condition, you can block a request in the Playground by setting Application Name to my-app or Model to gpt-4o-mini. After entering the values, click Send.

The response at the bottom of the page indicates that the request was blocked by the access rule.

Example response for a request blocked by an access rule

{
  ...
  "status": "Success",
  "summary": "Block my-app matched and blocked.",
  "result": {
    "blocked": true,
    "recipe": "my-app-input-policy",
    "detectors": {},
    "access_rules": {
      "block_my_app": {
        "matched": true,
        "action": "blocked",
        "name": "Block my-app"
      }
    },
    "input_token_count": 1,
    "output_token_count": 1
  }
}

For more comprehensive testing of access rules, deploy your collector in your application environment and send requests with the desired attribute values, as described in the Collectors documentation.

Test Prompt Rules

In the Text to guard field, enter a sample request that should trigger a prompt rule defined in the Input Policy selected in the top right.

The response at the bottom of the page shows details about any detections. In this example, it indicates that the request was blocked by a prompt rule.

Response for request blocked by a prompt rule

{
  ...
  "status": "Success",
  "summary": "Malicious Prompt was detected and blocked.  Confidential and PII Entity was not detected.  Malicious Entity was not executed.",
  "result": {
    "blocked": true,
    "recipe": "my-app-input-policy",
    "detectors": {
      "malicious_prompt": {
        "detected": true,
        "data": {
          "action": "blocked",
          "analyzer_responses": [
            {
              "analyzer": "PA4002",
              "confidence": 0.9296875
            }
          ]
        }
      },
      "confidential_and_pii_entity": {
        "detected": false,
        "data": {
          "entities": null
        }
      }
    },
    "access_rules": {
      "block_my_app": {
        "matched": false,
        "action": "allowed",
        "name": "Block my-app"
      },
      "report_suspicious_actor_or_location_when_data_is_sensitive": {
        "matched": false,
        "action": "allowed",
        "name": "Report suspicious actor or location when data is sensitive"
      }
    },
    "input_token_count": 28,
    "output_token_count": 28
  }
}

Similarly, by selecting the Output Policy in the top right of the Playground page, you can test how its rules report, block, or transform the model response.

Next Steps

• Learn more about collector types and deployment options in the Collectors documentation.
• On the Policies page in the AIDR console, configure access and prompt rules to align detection and enforcement with your organization’s AI usage guidelines.
• View collected data on the Visibility and Findings pages in the AIDR console. Events are associated with applications, actors, providers, and other context fields - and may be visually linked using these attributes.

Was this article helpful?

Contact us