Technology News

Tech for real-world applications: How enterprises can build AI systems tailored to their needs

Businesses are increasingly customizing large language models (LLMs) like GPT and Claude. Retrieval-Augmented Generation (RAG) and in-context learning offer distinct advantages.As artificial intelligence (AI) becomes a critical tool for businesses, organizations are grappling with how to best customize large language models (LLMs) like OpenAI's GPT and Anthropic's Claude to meet specific needs. A recent survey by Insight Partners found that nearly half of enterprises are adopting methods to incorporate industry-specific knowledge into these models. Traditionally, fine-tuning has been used to adapt AI models for tasks like image recognition or specialized business applications. However, newer techniques such as Retrieval-Augmented Generation (RAG) and in-context learning are gaining traction, offering distinct advantages for real-time and flexible AI performance. RAG and in-context learning: What's the difference? RAG, a method introduced by Meta AI researcher Patrick Lewis in 2020, allows LLMs to retrieve external data during use, making it particularly useful for businesses that require up-to-date information. For example, legal compliance and customer service tasks often demand precise and current data, making RAG an attractive option. In contrast, in-context learning provides examples or context directly within the task prompt, guiding the AI's responses without requiring external data retrieval. This approach relies on the pre-trained knowledge already within the model, making it simpler but less dynamic than RAG. The role of larger context windows Recent advancements in AI technology have expanded the capabilities of LLMs to handle much larger amounts of text at once. Leading models like Anthropic's Claude for enterprise can now process up to 500,000 tokens, while Google's Gemini 1.5 Pro and Gradient's AI startup have reached the million-token milestone. Efforts by companies like Google Cloud and Magic aim to push this limit to 100 million tokens, equivalent to about ten years of human speech. These breakthroughs raise questions about whether RAG will remain necessary. If LLMs can process vast amounts of information within their context windows, the need for external data retrieval might diminish. Challenges with larger context windows However, there are significant challenges to this approach. Research by Li et al. (2024) shows that models often struggle with lengthy, complex inputs. Information buried deep in a long sequence may be overlooked, even in models optimized for large context windows. Cost is another factor. Token-based pricing models mean that processing extensive text inputs can quickly become expensive for businesses. By keeping input lengths shorter, RAG offers a more cost-effective alternative. A hybrid future? While some experts predict that RAG could become less relevant as LLMs improve, new methods are blending the strengths of both approaches. Techniques like LongRAG, introduced by Jiang et al. (2024), group documents into larger units combining the efficiency of RAG with the expanded capabilities of long-context models. Additionally, innovations like Infini-Attention are making in-context learning more efficient by prioritizing the most critical information. Choosing the right approach Industry experts emphasize that RAG and long-context models have unique strengths: - Long-context models excel in tasks like multi-document summarization and long-term planning, where understanding extensive text is crucial. - RAG is better suited for real-time applications requiring precise and dynamic data retrieval, such as customer service and regulatory compliance. For many businesses, a hybrid solution may be the best option, leveraging RAG for efficiency and long-context models for depth. The path forward for enterprises Despite advances in long-context processing, RAG remains a practical and cost-effective option for many enterprises. A recent report by Harvard Business Review named RAG the leading method for building AI systems tailored to enterprise needs. As AI technologies continue to evolve, businesses must carefully evaluate their specific requirements and choose the methods -- or combinations -- that align with their goals. For now, both RAG and in-context learning remain vital tools in the race to customize AI for real-world applications. Devika Bhalla is a product strategist at a Fortune 500 technology company and a Fellow at the American Society of AI. The opinions expressed in this article are her own.

Latest Stories

Read Full Story