Using conceptual knowledge graph instead of a real knowledge graph with large language models

I found something curious

You can use an LLM to create a conceptual knowledge graph just as like a real knowledge graph without actually creating the knowledge graph itself - and then use that conceptual knowledge graph downstream just like a real knowledge graph

Let me explain with an example

Consider the scenario 

I want to model the following "create an optimal portfolio of stocks with the goal to increase the capital invested in the long term (3-5 years investment horizon) with a moderate risk tolerance" how could I do that

Using chatGPT the steps are 

1. Define Your Objective

2. Data Collection

3. Define Constraints and Assumptions

4. Select a Portfolio Optimization Method

5. Model Inputs

6. Optimization Setup

7. Backtesting and Validation

8. Rebalancing Strategy

For our case, considering long-term capital growth with moderate risk tolerance, the most relevant metrics would likely be:

• Sharpe Ratio: For risk-adjusted return evaluation.
• Sortino Ratio: To focus on downside risk.
• Volatility: To monitor overall risk.
• Maximum Drawdown: To limit severe losses.
• Value at Risk (VaR): To manage potential extreme losses.
• Alpha: To assess outperformance relative to a benchmark.
• Beta: To understand the portfolio’s sensitivity to market movements.

Now, I could ask

for the above machine learning problem, I want to create a requirements analysis document that ties to the model evaluation metrics identified above. how can I create such a requirements analysis document as a knowledge graph 

Then I could further say

from such a knowledge graph how can I create user stories and acceptance criteria? ensure that these are tied to the model evaluation metric

Summary of Process:

• Define Personas: Portfolio Manager, Risk Analyst, Investor.
• Write User Stories: Each story focuses on specific evaluation metrics like Sharpe Ratio, Sortino Ratio, Volatility, etc.
• Create Acceptance Criteria: Tie acceptance criteria directly to the formula and expected behavior of the evaluation metrics.
• Link to Knowledge Graph: Each story is linked back to entities and relationships in the knowledge graph, ensuring coherence between objectives, constraints, and metrics.

There are a number of advantages to using a knowledge graph such as a richer system for both complex decision making and for generation (such as generation of user stories and requirement analysis from knowledge graphs).

However, in this case, we have used chatGPT to create the user stories and acceptance criteria without explicitly creating the knowledge graph itself (ex in a graph database).

Instead, we referred to the conceptual structure of a knowledge graph and used its logical relationships (between objectives, constraints, metrics, etc.) to guide the creation of downstream ideas (in this case, user stories and acceptance criteria).

By using the mental model of a knowledge graph, we first discussed entities like Sharpe Ratio etc and their relationships, such as “Sharpe Ratio evaluates risk-adjusted return” or “volatility defines risk tolerance.” We then leveraged these conceptual relationships to write user stories and acceptance criteria without physically creating the knowledge graph itself.

The structure and relationships in the knowledge graph were mentally mapped and served as a reference point for capturing the requirements and connecting them to model evaluation metrics.

This approach allows you to focus on the logical connections and dependencies in your system - but does not necessitate the need to create the physical knowledge graph.

Notes:

1) You could still further enhance this output using a traditional RAG (to enrich the relationships and nodes based on an enterprise structure) without yet creating the knowledge graph itself.

2) I used chtGPT 4o. I could get better results using o1 for creation, generation and reasoning

This is indeed an interesting short cut which helps leverage the LLM rapidly by using a conceptual knowledge graph instead of a real knowledge graph. Welcome thoughts if anyone has similar results  

many thanks to Anjali Jain for her feedback