This is an interesting idea! Does it make it more costly to generate a response though? Since you need to make multiple api calls to chatgpt api and include the documents in most of them.
Not sure i understand this part: "... fetching relevant documents related to the document title"... So to clarify, are you fetching the relevant documents related to the user's query, then reranking the docs to figure out which to include in the final prompt? Or are you fetching additional documents based on the document title of initial documents retrieved based on the user's query?
This is an interesting idea! Does it make it more costly to generate a response though? Since you need to make multiple api calls to chatgpt api and include the documents in most of them.
Yes, that's one drawback of using this strategy. If I am right, ChatGPT's cost will go down significantly in the coming months.
Not sure i understand this part: "... fetching relevant documents related to the document title"... So to clarify, are you fetching the relevant documents related to the user's query, then reranking the docs to figure out which to include in the final prompt? Or are you fetching additional documents based on the document title of initial documents retrieved based on the user's query?