Chat completion with your own data

[eeadie]

Hi,

I have been following the example to use your own data within Azure OpenAI (https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/use-your-data?tabs=ai-search). Following this I uploaded a number of markdown files which creates an index for me.

Is it possible to configure the Kernel/chat completion (C#) to use the search index created from my data?

Thanks,

Euan

[dmytrostruk]

@eeadie Yes, here is an example of configuring Kernel and Chat Completion service to use with your data:
https://github.com/microsoft/semantic-kernel/blob/1e94588c06a3e39421f89547eeda5823c37ef9b5/dotnet/samples/Concepts/ChatCompletion/AzureOpenAIWithData_ChatCompletion.cs

[guru98]

Hi, you are right but that api have some limitation e.g. doesn't support plugin. Probably could be better create your own plugin and call search api from there

[eeadie]

@dmytrostruk thanks this looks useful. I have created an Azure AI search index, however I can't seem to find how to get the DataSourceEndpoint. Would you be able to point me towards any documentation that details how to get the endpont once setup?

[dmytrostruk]

@eeadie You can go to Overview page of your Azure AI Search service on Azure portal, there will be Url field, which contains a value that you need to use for DataSourceEndpoint property in code.

[guru98]

Is the endpoint of azure serach on azure. You can found it in azure portal

…

________________________________ Da: Euan Eadie ***@***.***> Inviato: lunedì 29 aprile 2024 11:14 A: microsoft/semantic-kernel ***@***.***> Cc: Ugo Castellani ***@***.***>; Comment ***@***.***> Oggetto: Re: [microsoft/semantic-kernel] Chat completion with your own data (Discussion #6016) @dmytrostruk<https://github.com/dmytrostruk> thanks this looks useful. I have created an Azure AI search index, however I can't seem to find how to get the DataSourceEndpoint. Would you be able to point me towards any documentation that details how to get the endpont once setup? — Reply to this email directly, view it on GitHub<#6016 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AA77TTU7EFSWFQBS3AY5PFLY7YFPDAVCNFSM6AAAAABG22SS76VHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TENRQGQ2TM>. You are receiving this because you commented.Message ID: ***@***.***>

[eeadie]

@dmytrostruk thnaks I managed to find the endpoint details on the Azure portal. Unfortunatley when invoking using chat completion GetSteamingChatMessageContentsAsync I am recieveing "Response status code does not indicate success: 404 (Resource Not Found)".

I am able to successfully interact with the Azure AI Search Index I created using VS Code REST client as suggested in this article (https://learn.microsoft.com/en-us/azure/search/search-get-started-rest). Any ideas why the call is not completing successfully?

[dmytrostruk]

@eeadie If Azure AI Search index works correctly when you test directly, could you check if Azure OpenAI ModelId and Endpoint are correct? ModelId is actually used as deployment name, so in case if in your setup deployment name and model id are different, you have to specify deployment name.

[eeadie]

@dmytrostruk the deployment name and model id are the same in my setup. The following config works (without data config):

When I switch to use config AzureOpenAIChatCompletionWithDataConfig I got the error mentioned above even though I can call Azure AI Search index directly without issue.

[guru98]

try tu use .AddAzureOpenAIChatCompletion(config: completionWithDataConfig) where in completionWithDataConfig you can configure all the values you try to use before.

[eeadie]

@guru98 I have already tried this, see the comment above

[ErikSkaarup]

Just for the record i see the same problem getting 404 using completionWithDataConfig, the individual services does work with the same credentials.

[dmytrostruk]

@eeadie @ErikSkaarup I'm going to investigate this issue and will back to you soon.

[dmytrostruk]

@eeadie @ErikSkaarup We added a new way how to specify with data configuration, here is an example:
https://github.com/microsoft/semantic-kernel/blob/3fdb1bb358d8acdf0c705d683e9330d93b3e8ff1/dotnet/samples/Concepts/ChatCompletion/AzureOpenAIWithData_ChatCompletion.cs

It should be released this week.

[eeadie]

@dmytrostruk many thanks for your help. I can confirm that is working for me using 1.14.1.

[eeadie]

@dmytrostruk unfortunatley I have hit an issue :(. What I am attempting to do is have a chat completion that has both plugins loaded and is able to query Azure AI Search index to access my own data (markdown files).

Using 1.14.1 I can get a chat completion to work using the Azure AI Search index I have created based on your example above.

However when I also attempt to import a plugin during the kernel setup (kernel.ImportPluginFromOpenApiAsync) as well as having the AzureChatExtensionsOptions config I am recieving the following error below. Removing the AzureChatExtensionsOptions allows the plugin to get hit as expected. Applying AzureChatExtensionsOptions and removing the plugin load allows the Azure AI Search index to be queried as expected.

Service request failed.
Status: 400 (Bad Request)

Content:
{"error": {"requestid": "c91e3a40-48cf-4a2e-9500-65d23b910834", "code": 400, "message": "Validation error at #/tools/0/function/description: String should have at least 1 characters\nValidation error at #/tools/1/function/description: String should have at least 1 characters\nValidation error at #/tools/2/function/description: String should have at least 1 characters\nValidation error at #/tools/3/function/description: String should have at least 1 characters"}}

Headers:
x-envoy-upstream-service-time: REDACTED
apim-request-id: REDACTED
x-ms-client-request-id: 72efc7e2-4af4-4e5d-81c4-1a5d2f653ef0
Strict-Transport-Security: REDACTED
X-Content-Type-Options: REDACTED
x-ms-region: REDACTED
x-ratelimit-remaining-requests: REDACTED
x-ratelimit-remaining-tokens: REDACTED
Date: Thu, 20 Jun 2024 09:07:11 GMT
Content-Length: 466
Content-Type: application/json; charset=UTF-8

[dmytrostruk]

@eeadie It looks like there is some issue with plugin validation, specifically with description field. But even if you provide valid descriptions for your plugins/functions, querying Azure AI Search index with function calling at the same time is not supported by Azure today:
https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/openai/Azure.AI.OpenAI/README.md#use-your-own-data-with-azure-openai

NOTE: The concurrent use of Chat Functions and Azure Chat Extensions on a single request is not yet supported. Supplying both will result in the Chat Functions information being ignored and the operation behaving as if only the Azure Chat Extensions were provided. To address this limitation, consider separating the evaluation of Chat Functions and Azure Chat Extensions across multiple requests in your solution design.

[eeadie]

@dmytrostruk many thanks for the advice. I have manged to get this working now.