I'd like to share thoughts on the high level process I am developing for use case discovery and deployment of Generative AI solutions. This is just a "brain dump" based on my hands on experience over the past few months. Please comment and share your thoughts on any of these things! This week at the NoVA Cyber Meetup we are going to try ethically and responsibly use an uncensored Mistral 7B open source LLM for offensive Cyber.
Assemble an AI working group including executive sponsor, line of business, Legal, HR Finance, Security and Operations representatives.
Find AI experts that can help you develop your AI solution stack.
Evaluate, prioritize and document use cases
Write test prompts and review with the group
Select and configure models (open source, Cloud, other)
Select inference engine (local or Cloud based)
Tune the inference engine (prompt templates, fine-tuning models, context sizing, prompt temperature, symmetry and other items)
Expose API’s to apps
Configure apps and Retrieval Augmented Generation pipelines (RAG)
Deploy Dev environment
Run uses cases and evaluate results
Fine tune the model(s) and test results
Develop Legal and Cyber frameworks to govern usage of the Gen AI solution and apps (for customers and employees)
Set prompt and response filtering and run guardrail tests
Size GPU's and hosting environment (on premise or Cloud)
Deploy with Kubernetes
Develop RBAC policies & sysadmin skills to run the AI stack
Deploy Observability solution
Run load testing and system performance benchmarking
Run model safety and bias checks as are applicable with your Legal & HR policies
Design and deploy SIEM integration and alerting methods for chat prompt and response collection
Financial modeling for running the use cases in Dev and Production
Review all results with AI Governance team throughout the process
Identify business risks arising from unpredictable model responses (Hallucinations).
Deploy a pilot to customers and employees and evaluate the results
Add more steps along the way to build in solution effectiveness, safety, security and compliance.
Get plenty of rest and fluids. AI is a marathon, not a sprint…
https://lnkd.in/esBHJqK4