Blending Chatbots with Large Language Models: A Guide to Architectural Choices

The launch of Chat GPT by OpenAI has created excitement in many organisations, offering glimpses into the vast possibilities of chatbots for answering common questions. It's tempting to assume that one can simply direct a Large Language Model (LLM) to a comprehensive knowledge base. However, this approach can risk losing control over responses, especially when questions overlap across multiple datasets.

Here, we explore different chatbot architectural options, highlighting their nuances, and their associated advantages and disadvantages.

Option 1: Traditional Chatbot Solutions (e.g., AWS Lex, Microsoft BOT framework, IBM Watson)

These platforms involve defining intents, entities, and utterances to develop conversational flows.

Pros:

• Offer natural language understanding
• Structured responses for FAQs

Cons:

• Time-consuming due to design work
• Customisation challenges
• Data set preparation complexities

Option 2: LLM Integration with Existing Chatbot Platforms

This involves an amalgamation of chatbot's understanding with LLM's expansive knowledge. For instance, upon receiving a query, the chatbot platform may forward complex queries to the LLM.

Pros:

• Comprehensive and nuanced responses
• Benefits from both pre-defined logic and LLM's extensive training

Cons:

• Requires a more nuanced Chatbot Developer skillset.

Figure 1: Example architecture | LLM integrated with Amazon Lex

Option 3: Standalone LLM with Bespoke Chatbot Interface

Directly interfaces the LLM with a customised user interface, bypassing intermediary platforms.

Pros:

• Seamless conversational experience
• Direct and contextually relevant responses

Cons:

• Lacks intermediary platforms' advanced features

Open Source Models: Although OpenAI often emerges as a top choice, the world of LLMs is vast and evolving. There's a surge in open-source LLMs. Models such as BAAI’s Aquila, EleutherAI’s GPT-J, Google’s Flamingo, and TII’s Falcon LLM are some notable names from a growing list. The open-source ethos is alluring, especially with platforms like Hugging Face serving as repositories. Such platforms often provide fine-tuned versions of foundational LLMs.

Addressing Data Privacy: When using an LLM, one might be concerned about data security. OpenAI's recent Azure service emphasises data privacy, assuring that data is not used to refine models, stored only for 30 days, and remains isolated from third-party access. For detailed insights, you might want to visit OpenAI's data privacy page.

Conclusion

Choosing the right chatbot architecture hinges on an organisation's specific needs. For those already equipped with chatbots, a full shift can be cumbersome and risky. However, incorporating an LLM can notably elevate response quality and expand its capabilities. The allure of open-source models is undeniable, yet understanding the implications for data privacy remains vital. The key lies in striking a balance between leveraging innovation and maintaining practical control.