make it a content for ppt:ServiceNow takes several measures to ensure the resiliency and…
- User
make it a content for ppt:ServiceNow takes several measures to ensure the resiliency and data integrity of its GenAI systems against AI-related vulnerabilities such as data poisoning, model extraction, and other attacks. Here are the key strategies and safeguards:
Trusted Security Program: ServiceNow has built a trusted security program for AI/ML data that is designed around the same principles applied to all customer instances in accordance with their Data Security Addendum. This program includes rigorous security protocols to protect against vulnerabilities.
Data Protection: Data collected for training is stored in secure AI development environments with access granted only to authorized users. This limited access helps to prevent unauthorized modifications that could lead to data poisoning. ServiceNow ensures that customer data is isolated and not co-mingled with other customers' data during processing as part of the subscription service. Customers also have the option to redact PII in prompts by configuring ServiceNow’s Sensitive Data Handling Plugin.
Continuous Monitoring and Testing: ServiceNow continuously tests the system to understand potential undesired content generated by the models. This includes conducting UX studies and evaluating model outputs for biases and vulnerabilities. The models are regularly updated and calibrated on a per-use case basis, incorporating feedback and new data to maintain their integrity and performance.
Model Evaluation and Bias Mitigation: Models undergo evaluation using both automated and human-in-the-loop feedback mechanisms. This helps in identifying and mitigating biases or vulnerabilities that could be exploited .
Secure Infrastructure: ServiceNow runs the models on its trusted infrastructure governed by the ServiceNow security model. This infrastructure does not normally require connections to third parties, reducing the risk of external attacks.
Ethical AI Practices: ServiceNow has an ethics charter that defines the values and ethical principles committed to in AI applications. These principles guide the design and deployment of AI systems to ensure they are used responsibly and securely.
Incident Response: ServiceNow leverages existing incident response processes to handle any incidents reported by customers. These processes include predefined service level agreements (SLAs) based on the priority and urgency of the incident.
- Assistant
Slide 1: Title Slide
ServiceNow: Resilience and Data Integrity in GenAI Systems
Ensuring Robustness Against AI-Related Vulnerabilities
Slide 2: Overview
Key Strategies and Safeguards for GenAI Systems
- Trusted Security Program
- Data Protection
- Continuous Monitoring and Testing
- Model Evaluation and Bias Mitigation
- Secure Infrastructure
- Ethical AI Practices
- Incident Response
Slide 3: Trusted Security Program
- Implementation of a trusted security program for AI/ML data.
- Designed around principles applied to all customer instances in line with the Data Security Addendum.
- Incorporation of rigorous security protocols to guard against vulnerabilities.
Slide 4: Data Protection
- Training data stored in secure AI development environments with limited access for authorized users.
- Prevents unauthorized modifications that could lead to data poisoning.
- Customer data is isolated during processing to prevent co-mingling.
- Option to redact PII using ServiceNow’s Sensitive Data Handling Plugin.
Slide 5: Continuous Monitoring and Testing
- Ongoing testing to detect undesired content generated by models.
- Includes UX studies and evaluations of model outputs.
- Regular updates and calibrations on a per-use case basis to enhance integrity and performance.
Slide 6: Model Evaluation and Bias Mitigation
- Models evaluated through both automated processes and human feedback.
- Focus on identifying and mitigating biases and vulnerabilities that could be exploited.
Slide 7: Secure Infrastructure
- Models run on trusted infrastructure governed by ServiceNow's security model.
- Reduced risk of external attacks as the infrastructure does not require third-party connections.
Slide 8: Ethical AI Practices
- Adherence to an ethics charter that defines values and principles in AI applications.
- Commitment to designing and deploying AI systems responsibly and securely.
Slide 9: Incident Response
- Use of established incident response processes to address customer-reported incidents.
- Incorporation of predefined SLAs based on the priority and urgency of incidents.
Slide 10: Conclusion
Ensuring Resiliency and Integrity
- ServiceNow employs a comprehensive approach to counteract AI-related vulnerabilities.
- Ongoing commitment to security, monitoring, and ethical practices enhances trust in GenAI systems.
Slide 11: Questions & Discussion
Thank You!
Questions or insights about our GenAI security measures?
Feel free to customize each slide with visuals or additional details as needed for your presentation!
- User
make all these summarized in one content.I don't need these many slides
- Assistant
ServiceNow: Ensuring Resiliency and Data Integrity in GenAI Systems
ServiceNow is committed to protecting the resilience and data integrity of its GenAI systems against various AI-related vulnerabilities, such as data poisoning and model extraction. Key strategies include:
-
Trusted Security Program: A rigorous security program for AI/ML data aligned with customer data security principles, ensuring robust protection against vulnerabilities.
-
Data Protection: Training data is stored in secure environments with limited access for authorized users, preventing unauthorized modifications. Customer data is isolated during processing, and users can redact PII through the Sensitive Data Handling Plugin.
-
Continuous Monitoring and Testing: ServiceNow regularly tests models to identify undesired outputs and conducts UX studies to evaluate biases. The models are continuously updated to maintain integrity and performance.
-
Bias Mitigation: Implementing both automated processes and human feedback to evaluate and mitigate biases in the models.
-
Secure Infrastructure: Models run on a trusted infrastructure governed by ServiceNow’s security protocols, minimizing external attack risks due to limited third-party connections.
-
Ethical AI Practices: Adhering to an ethics charter that guides the responsible and secure design and deployment of AI systems.
-
Incident Response: Utilizing established incident response processes to address customer-reported incidents with predefined service level agreements (SLAs) based on urgency.
Through these comprehensive measures, ServiceNow ensures its GenAI systems remain secure and reliable, fostering trust among its customers.
-
- User
For certain Now Assist skills, ServiceNow may make limited use of Azure OpenAI services through a ServiceNow license (at no additional cost to customers). For certain skills such as Skill Discovery, Skill Execution (data collector), Catalog Request Creation in Virtual Agent or Skill Discovery, Skill Execution (data collector), and Follow-up in Now Assist Panel, ServiceNow leverages third-party LLMs (Azure Open AI through a ServiceNow license) to augment Now LLM and power Now Assist capabilities.
ServiceNow’s Generative AI Controller supports customers who choose to configure and BYOL customer licenses to LLM services for OpenAI, Azure OpenAI, and Google Bard models. These capabilities are available in Process Automation Designer flows, Virtual Agent topics, and scripting like background scripts and business rules.
- Assistant
ServiceNow: Integration of Azure OpenAI Services in Now Assist
ServiceNow enhances its Now Assist capabilities by making limited use of Azure OpenAI services through a ServiceNow license at no additional cost to customers. This integration enriches specific functionalities, including:
- Skill Discovery
- Skill Execution (data collector)
- Catalog Request Creation in Virtual Agent
- Follow-up in the Now Assist Panel
By leveraging third-party large language models (LLMs), specifically Azure OpenAI, ServiceNow augments its capabilities, enhancing user interactions and service efficiency.
Furthermore, ServiceNow's Generative AI Controller empowers customers to configure and bring their own licenses (BYOL) for LLM services from OpenAI, Azure OpenAI, and Google Bard. These advanced capabilities can be utilized in various contexts, including:
- Process Automation Designer flows
- Virtual Agent topics
- Scripting, such as background scripts and business rules
This strategic integration allows ServiceNow to offer robust, flexible, and augmented AI-driven solutions for its users, improving productivity and customization options.
- User
summarize this:"For certain Now Assist skills, ServiceNow may make limited use of Azure OpenAI services through a ServiceNow license (at no additional cost to customers). For certain skills such as Skill Discovery, Skill Execution (data collector), Catalog Request Creation in Virtual Agent or Skill Discovery, Skill Execution (data collector), and Follow-up in Now Assist Panel, ServiceNow leverages third-party LLMs+E33:E34 Please see the data and usage policies for the respective third-party LLMs to learn more. (https://docs.servicenow.com/bundle/vancouver-intelligent-experiences/page/administer/generative-ai-controller/reference/reference-for-generative-ai-controller.html#d13487e359)" "Traffic from customer instances to ServiceNow GenAI regional compute hub passes through ServiceNow backbone network with MACSec. All GenAI requests go through the GenAI controller application which is installed on each individual customer instance and supports the construction and routing of the prompt and response data, which requires entitlement. Access to the GenAI backend requires mTLS, limiting access to trusted ServiceNow components. mTLS is managed by ServiceNow.
Use of other 3rd party AI is configured by the customer, traffic passes from their instance to the 3rd party service across the Internet. Integration protocols are configured by the customer on glide including TLS version." "ServiceNow aims to develop AI solutions and provide tools that enhance customer control and understanding. Our tools and interfaces are built to ensure transparency, traceability, and the ability for customers to challenge and understand the AI's impact on their operations. Instead of dictating actions, Now Assist presents options, maintaining customers’ independence in making decisions. ServiceNow's AI products provide methods to review outputs, provide feedback, and, if necessary, adjust AI-generated decisions, prioritizing autonomy and informed choice.
The ServiceNow AI governance policies define an Artificial Intelligence Software Development Lifecycle, and provide guidance for each stage of the product development lifecycle, including functional testing, skill-specific testing, performance testing, safety testing, security testing, and post-deployment monitoring and analysis. Policies specify the need for ongoing product review to evaluate the performance of the AI system against the model's stated goals and benefits.
ServiceNow offers Responsible/Human-Centered AI Assessments to lead in establishing, managing, and expanding governance frameworks, educational resources, documentation, and methods that empower customers to use AI responsibly and prevent misuse. See Responsible/Human-Centered AI Assessments: https://www.servicenow.com/standard/other-documents/responsible-ai-guidelines.html" "ServiceNow maintains a documented AI governance policies and standard procedures includes industry-standard practices, e.g. AI data gavornance, dataset lifecycle management, AI development, AI deployment, etc. ServiceNow updates the policy to address new and evolving technologies, changes to industry standard practices, and changing threats.
The ServiceNow AI Governance process aligns with the NIST 2.0 RMF for AI Framework and has established an oversight & governance committee overseeing Responsible AI & AI Governance efforts across all functions. This Internal governance body oversees day to day product and development operations, ensuring customers can utilize ServiceNow's AI capabilities with confidence. ServiceNow works closely with external experts and the AI community to provide feedback.
ServiceNow complies with all applicable local laws and regulations, which are constantly evolving. The company continues to adopt, expand, and align with compliance programs and regulations as applicable to ensure ongoing compliance.
See “Responsible AI Whitepaper” - https://www.servicenow.com/content/dam/servicenow-assets/public/en-us/doc-type/resource-center/white-paper/wp-sn-responsible-genai.pdf
Finally, ServiceNow has completed a SOC2 Type 2 assessment for the AI/ML product lines, and customers/prospects under NDA will be able to review it upon publication, which is expected to be released in early 2025."
- Assistant
Summary of ServiceNow's AI Integration and Governance
ServiceNow incorporates Azure OpenAI services, at no extra cost to customers, to enhance specific Now Assist skills—such as Skill Discovery, Skill Execution, and Catalog Request Creation within the Virtual Agent—by leveraging third-party large language models (LLMs). Customers can refer to relevant data and usage policies for these models for more information.
All GenAI requests are routed through the GenAI controller application installed on individual customer instances, which facilitates prompt and response data management with secure access via mutual TLS (mTLS). For other third-party AI services, customer-configured traffic flows over the internet, using specified integration protocols.
ServiceNow aims to empower customers by developing AI solutions that emphasize transparency and control. Instead of dictating actions, Now Assist provides options to ensure customer autonomy, allowing for feedback and adjustments to AI-generated decisions. The company follows a comprehensive AI governance policy that includes structures for ongoing evaluation and performance monitoring within an Artificial Intelligence Software Development Lifecycle.
By aligning with NIST 2.0 RMF for AI Framework, ServiceNow maintains an internal governance body that oversees Responsible AI efforts. The organization continually updates its AI governance policies to address technological changes and industry standards while ensuring compliance with local laws and regulations.
To further promote responsible AI usage, ServiceNow offers Responsible/Human-Centered AI Assessments and collaborates with external experts. They have also completed a SOC2 Type 2 assessment for their AI/ML product lines, with a report expected for review by customers under NDA in early 2025. For more details, resources like the Responsible AI Whitepaper are available on ServiceNow's website.
- User
make it content of a slide
- Assistant
ServiceNow's AI Integration and Governance
-
Azure OpenAI Services Integration:
- Leverages third-party LLMs to enhance Now Assist skills (e.g., Skill Discovery, Skill Execution, Catalog Request Creation) at no additional cost.
- See usage policies for more details.
-
Secure AI Request Management:
- All GenAI requests are managed through a GenAI controller on individual customer instances with mutual TLS (mTLS) for secure access.
- Traffic to third-party AI services is customer-configured over the internet.
-
Customer Empowerment:
- Emphasis on transparency and control, allowing customers to make informed decisions with Now Assist.
- Capabilities for feedback and adjustments on AI-generated outputs.
-
AI Governance Framework:
- Follows an AI Software Development Lifecycle with ongoing performance evaluation and monitoring.
- Aligns with NIST 2.0 RMF for AI Framework and maintains an internal governance body for Responsible AI efforts.
-
Compliance and Responsible AI:
- Adapts policies to meet evolving technologies and regulations.
- Offers Responsible/Human-Centered AI Assessments to support ethical AI use.
- Completed SOC2 Type 2 assessment; report available for review in early 2025.
-
Resources:
- For more information, refer to the Responsible AI Whitepaper.
-
- User
ServiceNow ensures AI/ML model transparency via model cards highlighting a given model’s capabilities and limitations. We also assess risks and impacts that may result from a reasonable implementation of our AI products and services; for our current suite of AI products, we do not consider them high-risk if used as designed. With respect to Explainability, we offer context-specific explanations within our workflows. For instance, Now Assist for AI Search provides references to KB articles along with the generated results, which enhances customer confidence in the solution. ServiceNow also has a Trustworthy Lab for Explainability. In the future, if applicable, ServiceNow will use additional Explainability methods. ServiceNow logs all activities with timestamps which includes workflow steps and also calls to the LLMs. This can be audited as required, however depending on the user who is doing the auditing the roles & permissions may need to be configured appropriately. Some of the tables are locked down from access to the end-users, for security purposes. same for this
- Assistant
ServiceNow's Commitment to AI/ML Model Transparency
-
Model Cards:
- Provide clear insights into each AI/ML model’s capabilities and limitations.
-
Risk Assessment:
- Evaluate risks and impacts associated with the implementation of AI products.
- Current AI products are deemed low-risk when used as designed.
-
Explainability:
- Context-specific explanations are integrated into workflows.
- Example: Now Assist for AI Search offers references to knowledge base (KB) articles alongside generated results to enhance customer confidence.
-
Trustworthy Lab:
- Established for ongoing research and development of Explainability methods. Additional methods may be adopted in the future if needed.
-
Activity Logging:
- All activities, including workflow steps and LLM calls, are logged with timestamps for audit purposes.
- Access to logs may require appropriate roles and permissions due to security restrictions; certain tables are locked down to protect sensitive information.
This comprehensive approach underlines ServiceNow's dedication to maintaining transparency and trust in its AI and ML solutions.
-
- User
"Yes, ServiceNow performs rigerous testing as part of our release and deployment processes on all areas of the platform. Additional information related to ServiceNow's internal Standard Operating Procedures should be available through ServiceNow CORE (via self-service) "
- Assistant
ServiceNow's Testing and Quality Assurance
-
Rigorous Testing:
- Comprehensive testing is conducted across all areas of the platform as part of the release and deployment processes.
-
Standard Operating Procedures:
- Additional details regarding ServiceNow’s internal Standard Operating Procedures (SOPs) can be accessed through ServiceNow CORE via self-service.
This commitment to thorough testing ensures high-quality and reliable performance of ServiceNow products and services.
-
- User
create a content based on this:"Each product we build undergoes a detailed review for compliance with our Human-Centered AI principles and standards for safety.
Additionally, where appropriate, safeguards like filters are implemented in our AI products to help ensure safe interactions by blocking harmful content.
We encourage users to ""check for correctness"" whenever content is generated. Additionally, our designs include informative tooltips to offer further insights into the capabilities of the system.
Moreover, to ensure the continuous improvement and safety of our AI technology, we place immense value on human feedback in our product development process. For example, we have implemented some user-friendly feedback mechanisms to facilitate easy human participation. Additionally, we engage in ongoing UX research with our customers and their users, employing diverse feedback methods to inform and refine our product experiences.
You may refer to Responsible AI whitepaper (https://www.servicenow.com/content/dam/servicenow-assets/public/en-us/doc-type/resource-center/white-paper/wp-sn-responsible-genai.pdf) which highlights ServiceNow's approach to ethical AI. It outlines the major themes of ServiceNow's commitment to human-centric, inclusive, transparent, and accountable AI practices, ensuring customers understand the core principles that drive the development and deployment of AI solutions. " "Yes, the principle of transparency is a fundamental aspect of our AI UX design methodology, and we believe in increasing clarity and understanding for all users. One example of this principle is to ensure that we visually distinguish between responses generated by AI and those created by human agents (e.g. the sparkle in code editors where genAI is enabled).
In addition to product-specific documentation and usage guidelines, we also provide AI-specific documentation, ""Model Cards,"" where appropriate. "
"The Now Platform provides the data used to generate AI responses. For example, Case summarization uses field data from the Case table. AI Search uses data found in the Knowledge Base and Service Catalog. Now Assist then generates text responses such as summaries, Q&A answers, and resolution notes. The input data used for each type of interactions varies, but is visible within the system configuration. An example of the Incident Summarization skill is available at our docs site, and shows the specific fields used in the incident summarization skill. When activating this skill, some of these values are configurable by the customer. Details can be found here: https://www.servicenow.com/docs/bundle/xanadu-intelligent-experiences/page/administer/now-assist-admin/task/configure-case-or-incident-summarization-in-the-now-assist-admin-console.html Another example is the Now Assist Virtual Agent configuration which allows the customer to define the knowledge bases and articles that are available to the virtual agent to leverage in generating its responses. Now Assist leverages our in-platform AI Search capability as part of a retrieval augmented generation (RAG) process to find relevant information from the customers platform (using the customer-defined sources) which it then augments the prompt sent to the LLM to generate its responses.
With respect to the responses from GenAI, these are all labelled clearly as something generated by AI and that the answers should be checked by the user. This is flagged on the Virtual Agent console at the start of each interaction and also labelled on any generated summarizations presented to the user (as well as all other occassions) in the name of transparency" "If used as designed, ServiceNow use cases are not considered high risk per initial analysis based on the criteria set forth in the EU act.
We minimize the risk for harm by grounding the responses of generative AI in the knowledge contained in customers’ knowledge base or in agent conversations and by putting safeguards in place designed to generative answers from that content base and at detecting harmful/undesired content in GenAI answers. We also design user interfaces that communicate AI uncertainty and encourage our users to verify the accuracy of AI-generated responses. Our products are not designed to answer general questions outside of what can be reasonably asked and answered by the customer.
Furthermore, ServiceNow LLM model cards (https://docs.servicenow.com/csh?version=latest&topicname=now-llm-model-updates) explain the context, intended use, training data, risks, and limitations, along with other important information related to the model powering some of our AI products.
Finally, each of the use-cases that are leveraging GenAI may have its own risk profile and consequences and would need to be reviewed independently. This applies to OOTB provided content and probably moreso to any use-cases that the customer may choose to build on our platform too." The Now Platform provides the data used to generate AI responses. (i.e. Standard Chartered Bank's data stored on Now Platform, in your instance.)
- Assistant
ServiceNow's Commitment to Human-Centered AI Principles and Transparency
-
Compliance and Safety:
- Each product undergoes a rigorous review to ensure compliance with Human-Centered AI principles and safety standards.
-
Safeguards and Content Filters:
- AI products incorporate safeguards, such as content filters, to block harmful content and facilitate safe interactions.
-
User Engagement:
- Users are encouraged to "check for correctness" on generated content, with informative tooltips provided to clarify the system's capabilities.
-
Continuous Improvement Through Feedback:
- Value is placed on human feedback in the product development process.
- User-friendly feedback mechanisms and ongoing UX research are employed to refine product experiences based on diverse customer insights.
-
Transparency in AI Responses:
- The design methodology includes features that clearly distinguish between AI-generated responses and those by human agents (e.g., visual markers in interfaces).
- Documentation such as AI-specific "Model Cards" is provided to describe models' capabilities and limitations.
-
Data Utilization:
- The Now Platform supplies the data for generating AI responses (e.g., Case summarization uses data from the Case table; AI Search pulls from the Knowledge Base and Service Catalog).
- Examples of configurations, such as Incident Summarization skills and Now Assist Virtual Agent settings, allow customers to define their knowledge sources for enhanced response accuracy.
-
Transparency in AI Outputs:
- All AI-generated responses are clearly labeled, alerting users that the answers should be verified for accuracy. This message is visible throughout interactions, such as in the Virtual Agent console.
-
Risk Management:
- Current use cases, when used as intended, are not deemed high risk according to EU regulatory criteria.
- ServiceNow minimizes potential harm by grounding AI responses in existing customer knowledge bases and implementing safeguards against unwanted content.
-
Model Cards:
- Model cards provide insights into the context, intended use, training data, risks, and limitations of the AI models driving ServiceNow's products.
-
Independent Risk Profiles:
- Each use case involving generative AI is assessed individually for risk factors and consequences, both for out-of-the-box content and custom use cases developed by customers.
-
Resources:
- For further details, refer to the Responsible AI Whitepaper and LLM Model Cards.
-
- User
ServiceNow's approach is a combination of the model development and training approach and also to use the customer's data via Retrieval Augmented Generation to ensure that the responses from the model are heavily grounded to avoid hallucinations. Additionally, by using constrained, focused data sets, we’re also able to carry out comprehensive, rigorous testing—improving model quality and lowering risk Yes, ServiceNow collects metrics and user feedback post-release, including in-product feedback to identify issues and drive a cycle of continual improvement for each LLM.
Feedback on the responses from the model can also be evaluated by the user (using Thumbs up/down) which can be used to detect and report on the quality of the responses from the model.
- Assistant
ServiceNow's AI Model Development and Continuous Improvement Strategy
-
Model Development and Training:
- ServiceNow employs a dual approach that combines robust model development and training with the effective use of customer data through Retrieval Augmented Generation (RAG). This strategy ensures that model responses are grounded in reliable information, significantly minimizing the risk of hallucinations.
-
Use of Focused Data Sets:
- By utilizing constrained and focused data sets, ServiceNow conducts comprehensive testing. This process enhances model quality and reduces associated risks.
-
Metrics and Feedback Collection:
- Post-release, ServiceNow collects metrics and user feedback to identify areas for improvement and foster continuous development of each language model (LLM).
-
User Engagement in Quality Assessment:
- Users can evaluate model responses using a thumbs up/down system, allowing for real-time detection and reporting of response quality. This feedback mechanism contributes to ongoing enhancements and ensures that the model continues to meet user needs effectively.
Through these methodologies, ServiceNow is committed to delivering high-quality AI interactions while continuously refining its models based on user experiences and data-driven insights.
-
- User
draft a mail attaching a ppt for review for RAI forum
- Assistant
Subject: Request for Review of RAI Forum Presentation
Dear [Recipient's Name],
I hope this message finds you well.
I am writing to share a PowerPoint presentation that I have prepared regarding our approach to Responsible Artificial Intelligence (RAI). The presentation outlines our key initiatives, principles, and strategies aimed at ensuring ethical and transparent AI practices.
I would greatly appreciate it if you could take some time to review the attached PPT and provide your valuable feedback. Your insights will be instrumental in refining our approach and ensuring that we effectively communicate our commitment to responsible AI usage.
Please feel free to share any suggestions or areas for improvement. The deadline for feedback is [insert deadline], as I plan to finalize the presentation shortly thereafter.
Thank you for your time and support. I look forward to hearing your thoughts!
Best regards,
[Your Name]
[Your Job Title]
[Your Contact Information]
[Your Company]Attachment: RAI_Forum_Presentation.pptx