Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

addition of Amazon Bedrock Guardrail support for the case summarization #267

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

addition of Amazon Bedrock Guardrail support for the case summarization #267

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 60 additions & 12 deletions case-summarization/deploy/case-summarization.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
AWSTemplateFormatVersion: '2010-09-09'
Description: AWS Support Cases Summarization Stack v0.0.1
Description: AWS Support Cases Summarization Stack v0.0.2
Metadata:
AWS::CloudFormation::Interface:
ParameterGroups:
Expand All @@ -16,6 +16,9 @@ Metadata:
- MaxRetries
- Timeout
- BatchSize
- GuardRailIdentifier
- GuardRailVersion
- GuardRailTrace
- Label:
default: 'Technical parameters'
Parameters:
Expand Down Expand Up @@ -78,6 +81,21 @@ Parameters:
Type: String
Description: Number of processing in parallel. Use 1 to avoid Throttling.
Default: '1'
GuardRailIdentifier:
Type: String
Description: The identifier for the guardrail. Leave empty if you do not want to use Amazon Bedrock Guardrails.
Default: ''
GuardRailVersion:
Type: String
Description: The version for the guardrail. Leave empty if you do not want to use Amazon Bedrock Guardrails.
Default: ''
GuardRailTrace:
Type: String
Description: The trace behavior for the guardrail.
AllowedValues:
- ENABLED
- DISABLED
Default: 'ENABLED'

Conditions:
LambdaLayerBucketPrefixIsManaged: !Equals [!Ref LambdaLayerBucketPrefix, 'aws-managed-cost-intelligence-dashboards']
Expand Down Expand Up @@ -163,7 +181,7 @@ Resources:
- Fn::Sub:
- "arn:${AWS::Partition}:s3:::${DestinationBucket}"
- DestinationBucket: {"Fn::ImportValue": "cid-DataCollection-Bucket"}
- PolicyName: "Bedrock-Access"
- PolicyName: "Bedrock-ModelAccess"
PolicyDocument:
Version: "2012-10-17"
Statement:
Expand All @@ -172,6 +190,14 @@ Resources:
- "bedrock:ListFoundationModels"
- "bedrock:InvokeModel"
Resource: "*" # cannot restrict this
- PolicyName: "Bedrock-GuardrailAccess"
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: "Allow"
Action:
- "bedrock:ApplyGuardrail"
Resource: !Sub "arn:${AWS::Partition}:bedrock:${BedrockRegion}:${AWS::AccountId}:guardrail/*"
- PolicyName: "SQS-Access"
PolicyDocument:
Version: "2012-10-17"
Expand Down Expand Up @@ -223,6 +249,9 @@ Resources:
MAX_RETRIES = os.environ.get("MAX_RETRIES", 30)
TIMEOUT = os.environ.get("TIMEOUT", 60)
INSTRUCTIONS = os.environ.get("INSTRUCTIONS", '')
GUARDRAIL_ID = os.environ.get("GUARDRAIL_ID", '')
GUARDRAIL_VERSION = os.environ.get("GUARDRAIL_VERSION", '')
GUARDRAIL_TRACE = os.environ.get("GUARDRAIL_TRACE", '')
PROMPT_TEMPLATE = f"""
System: You are an expert technical writer specializing in creating concise, neutral summaries of AWS customers support interactions. Your task is to summarize conversations between customers and AWS Support, maintaining objectivity and clarity. Here is the Conversation to be summarized:
<conversation>
Expand Down Expand Up @@ -261,15 +290,31 @@ Resources:
default=""
)

def get_llm_program(conversation, model_id):
llm = Bedrock(
model=model_id,
region_name=REGION,
temperature=float(TEMPERATURE),
max_tokens=int(MAX_TOKENS),
max_retries=int(MAX_RETRIES),
timeout=float(TIMEOUT)
)
def get_llm_program(conversation, model_id, guardrail_identifier, guardrail_version, trace):
if guardrail_identifier == '' or guardrail_version == '':
logger.info("support case summarization isn't using any Amazon Bedrock Guardrail Configuration.")
llm = Bedrock(
model=model_id,
region_name=REGION,
temperature=float(TEMPERATURE),
max_tokens=int(MAX_TOKENS),
max_retries=int(MAX_RETRIES),
timeout=float(TIMEOUT)
)
elif guardrail_identifier != '' and guardrail_version != '':
logger.info(f"support case summarization is using Amazon Bedrock Guardrail id: {guardrail_identifier} (Version {guardrail_version}) with tracing {trace}")
llm = Bedrock(
model=model_id,
region_name=REGION,
temperature=float(TEMPERATURE),
max_tokens=int(MAX_TOKENS),
max_retries=int(MAX_RETRIES),
timeout=float(TIMEOUT),
guardrail_identifier=guardrail_identifier,
guardrail_version=guardrail_version,
trace=trace
)

return LLMTextCompletionProgram.from_defaults(
llm=llm,
output_cls=Summary,
Expand Down Expand Up @@ -314,7 +359,7 @@ Resources:
logger.info("support case summarization starting")
model_id = get_model_id()

llm_program = get_llm_program(communications, model_id)
llm_program = get_llm_program(communications, model_id, GUARDRAIL_ID, GUARDRAIL_VERSION, GUARDRAIL_TRACE)
try:
case_data_content['Summary'] = llm_program().model_dump_json()
except Exception as exc:
Expand Down Expand Up @@ -353,6 +398,9 @@ Resources:
TIMEOUT: !Ref Timeout
INSTRUCTIONS: !Ref Instructions
REGION: !Ref BedrockRegion # FIXME: use default if empty
GUARDRAIL_ID: !Ref GuardRailIdentifier
GUARDRAIL_VERSION: !Ref GuardRailVersion
GUARDRAIL_TRACE: !Ref GuardRailTrace
Metadata:
cfn_nag:
rules_to_suppress:
Expand Down
81 changes: 81 additions & 0 deletions case-summarization/guardail/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
## AWS Bedrock Guardrail
Amazon Bedrock Guardrail is a crucial security feature for generative AI applications that helps implement safeguards based on specific use cases and responsible AI policies. It provides an additional layer of protection on top of the native safeguards offered by foundation models (FMs)[1][2].

## Key Features and Importance

Amazon Bedrock Guardrails offers several important security features:

1. **Content Filtering**: It helps block harmful content by evaluating both user inputs and model responses. The system can filter out content related to hate speech, insults, sexual content, violence, and misconduct[2].

2. **Topic Restrictions**: Organizations can define specific topics to avoid, ensuring that interactions remain relevant to their business and align with company policies[2].

3. **Sensitive Information Protection**: The system can detect and redact personally identifiable information (PII) in user inputs and model responses, helping to protect user privacy[2][3].

4. **Custom Word Filtering**: It allows the configuration of custom words or phrases to be blocked, including profanity or specific terms like competitor names[2].

5. **Hallucination Detection**: Contextual grounding checks help detect and filter out hallucinations in model responses, ensuring more accurate and trustworthy information[2].

## Security Importance

The importance of Amazon Bedrock Guardrails for security cannot be overstated:

1. **Enhanced Content Safety**: It can block up to 85% more harmful content compared to native FM protections, significantly improving the safety of AI applications[2].

2. **Consistent Security Across Models**: Guardrails work with all large language models in Amazon Bedrock, providing a uniform level of security regardless of the underlying model[2].

3. **Customizable Safeguards**: Organizations can create multiple guardrails with different configurations, tailoring security measures to specific applications and use cases[1][3].

4. **Compliance and Responsible AI**: By allowing fine-tuned control over content and interactions, Guardrails help organizations adhere to their responsible AI policies and maintain regulatory compliance[2].

5. **Protection Against Prompt Attacks**: The system safeguards against prompt injection and jailbreak attempts, enhancing overall security[2].

Amazon Bedrock Guardrails plays a vital role in ensuring that generative AI applications remain safe, relevant, and aligned with organizational policies. By providing robust, customizable security features, it enables businesses to leverage the power of AI while mitigating potential risks associated with harmful or inappropriate content[1][2][3].

### Reasonable Defaults

This plugin comes with the following reasonable defaults that can be overriden through the parameters exposed by the CloudFormation template:

| Parameter | Description | Default |
| --- | --- | --- |
| BlockedInputMessage | Message to return when the Amazon Bedrock Guardrail blocks a prompt. | {"executive_summary":"Amazon Bedrock Guardrails has blocked the AWS Support Case Summarization.","proposed_solutions":"","actions":"","references":[],"tam_involved":"","feedback":""} |
| BlockedOutputMessage | Message to return when the Amazon Bedrock Guardrail blocks a model response | '' |
| IncludeSexualContentFilter | Whether to include Sexual Content Filter in the Guardrail or not | 'yes' |
| SexualContentFilterInputStrength | The strength of the content filter to apply to prompts. As you increase the filter strength, the likelihood of filtering harmful content increases and the probability of seeing harmful content in your application reduces. | 'HIGH' |
| SexualContentFilterOutputStrength | The strength of the content filter to apply to model responses. As you increase the filter strength, the likelihood of filtering harmful content increases and the probability of seeing harmful content in your application reduces | 'HIGH' |
| IncludeViolentContentFilter | Whether to include Violent Content Filter in the Guardrail or not | 'yes' |
| ViolentContentFilterInputStrength | The strength of the content filter to apply to prompts. As you increase the filter strength, the likelihood of filtering harmful content increases and the probability of seeing harmful content in your application reduces | 'HIGH' |
| ViolentContentFilterOutputStrength | The strength of the content filter to apply to model responses. As you increase the filter strength, the likelihood of filtering harmful content increases and the probability of seeing harmful content in your application reduces | 'HIGH' |
| IncludeHateContentFilter | Whether to include Violent Content Filter in the Guardrail or not | 'yes' |
| HateContentFilterInputStrength | The strength of the content filter to apply to prompts. As you increase the filter strength, the likelihood of filtering harmful content increases and the probability of seeing harmful content in your application reduces | 'HIGH' |
| HateContentFilterOutputStrength | The strength of the content filter to apply to prompts. As you increase the filter strength, the likelihood of filtering harmful content increases and the probability of seeing harmful content in your application reduces | 'HIGH' |
| IncludeInsultsContentFilter | Whether to include Insults Content Filter in the Guardrail or not | 'yes' |
| InsultsContentFilterInputStrength | The strength of the content filter to apply to prompts. As you increase the filter strength, the likelihood of filtering harmful content increases and the probability of seeing harmful content in your application reduces | 'HIGH' |
| InsultsContentFilterOutputStrength | The strength of the content filter to apply to prompts. As you increase the filter strength, the likelihood of filtering harmful content increases and the probability of seeing harmful content in your application reduces | 'HIGH' |
| IncludeMisconductContentFilter | Whether to include Insults Content Filter in the Guardrail or not | 'yes' |
| MisconductContentFilterInputStrength | The strength of the content filter to apply to prompts. As you increase the filter strength, the likelihood of filtering harmful content increases and the probability of seeing harmful content in your application reduces | 'HIGH' |
| MisconductContentFilterOutputStrength | The strength of the content filter to apply to prompts. As you increase the filter strength, the likelihood of filtering harmful content increases and the probability of seeing harmful content in your application reduces | 'HIGH' |
| IncludePromptAttackContentFilter | Whether to include Insults Content Filter in the Guardrail or not | 'yes' |
| PromptAttackContentFilterInputStrength | The strength of the content filter to apply to prompts. As you increase the filter strength, the likelihood of filtering harmful content increases and the probability of seeing harmful content in your application reduces | 'HIGH' |

### References & Further reading

* [1] How Amazon Bedrock Guardrails works https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails-how.html
* [2] Generative AI Data Governance - Amazon Bedrock Guardrails - AWS https://aws.amazon.com/bedrock/guardrails/
* [3] Stop harmful content in models using Amazon Bedrock Guardrails https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails.html

## Usage

This stack will deploy a minimalistic Amazon Bedrock Guardrail that will filter out any inputs or outputs that can be assimilated to prompt hacking, sexual, violent, misconduct, hatred speech or insults. Any additional fine-tuning of filters can be acheived by customizing this template.

## Support and Contribution

See [CONTRIBUTING](../../../CONTRIBUTING.md) for more information.

## Security

See [SECURITY](../../../SECURITY.md) for more information.

## License

This project is licensed under the Apache-2.0 License.

Loading
Loading