.png)
.png)
RAG
RAG
LLM and RAG are generative AI technologies that have recently attracted significant attention from companies. Indeed, the combination of LLM and RAG demonstrates innovative potential compared to existing AI systems, and their application is growing in various industries. However, real-world challenges, such as performance issues and difficulties in maintaining continuous data updates, must be overcome to achieve investment returns. Daewon CTS collaborates with various partners to address these challenges, leveraging its LLM/RAG technology capabilities to provide comprehensive support, from consulting to optimization.
Trial and error,
Reducing is the key
More and more organizations are considering implementing LLM/RAG. However, many are hesitant. This is because implementation and operation can be costly, the ROI of LLM/RAG implementation is difficult to objectively measure, and the benefits are difficult for users to immediately perceive. Therefore, it's crucial to prepare in advance by examining all possible variables from the outset of the project to minimize trial and error.
Challenge
RAG system
challenges
Complexity of
data collection and cleansing
-
Difficulty in collecting internal enterprise text data of varying formats and quality
-
Significant time required for preprocessing and cleansing
-
Challenges in splitting documents into chunks after document segmentation
-
Complexity in converting each chunk into embedding vectors and building database indexes
-
Difficulty in determining appropriate chunk sizes and splitting strategies
-
Degradation of embedding vector reliability caused by noise, duplication, and contradictory content in source documents
The burden of prompt engineering and tuning
-
Prompt design, which is central to effective LLM usage, is not a one-time task but a process of continuous tuning.
-
In RAG systems, simply providing documents is insufficient to generate the desired responses.
-
Prompts must include structured instructions, such as prioritizing reference materials and citing sources.
-
Response tone, length, and format need to be guided through examples and explicit instructions.
-
Securing specialized personnel and establishing long-term maintenance plans are essential; without proper planning, identifying the causes of quality degradation becomes difficult.
The difficulty of continuous data updates
-
Difficulty in managing data update cycles
-
Operational burden caused by varying data refresh frequencies
-
Risk of information becoming outdated when updates are too infrequent
-
Challenges in implementing updates for domains where real-time data is critical
-
Technical difficulty in immediately reflecting new data or document changes in vector databases
-
Supporting continuous data updates requires end-to-end automation systems across data pipelines, vector databases, search indexes, and models, demanding significant expertise for both implementation and operations
Performance issues
-
As data volumes grow, latency in vector search can become a significant issue.
-
Search performance issues may arise as the number of embeddings stored in the vector database increases or as usage scales.
-
Maintaining performance becomes challenging when data domains evolve over time or when embedding models need to be upgraded.
Service
Optimization services
DIA NEXUS focuses on.
RAG performance
optimization
Select the right-sized language model for each use case, and optimize performance and cost through prompt engineering, continuous monitoring, and ongoing tuning strategies.
Continuous
data updates
Recommend a data-refresh workflow that keeps knowledge always current—DMS integration, automated embedding and trigger setup, batch updates, change history tracking, index optimization, and scheduled re-indexing.
Security and regulatory compliance
Propose customized security measures that meet internal corporate policies and industry regulations, including encryption, access control, and related safeguards.