Timestamped Embeddings for Time-Aware Retrieval-Augmented Generation (RAG)

“amethyst”

A crucial element of RAG is the use of embeddings — vector representations of text that capture semantic meaning. However, as knowledge evolves over time, the need for temporal awareness in these embeddings becomes essential.

This is where timestamped embeddings come into play, adding a time dimension to embeddings to ensure AI systems retrieve and process the most relevant and timely information.

This article explores timestamped embeddings, their significance in RAG applications, and how they can be implemented.

Why Timestamped Embeddings Matter?

Without incorporating temporal data into embeddings, a language model lacks sufficient context to determine when the events represented by these embeddings occurred, unless explicitly stated. During the vectorization of knowledge bases, financial documents, articles, and similar materials, dates and times are rarely included in each segment.

For example, our annual summary for 2024 will typically not have the phrase 'annual summary 2024 written in December 2024' in every part. Instead, segments contain only the report's content, which, while understandable, does not assist the language model in discerning timing.

The query produces segments that include the term 'annual report,' but omit those that contain the detailed content of the report. As a result, the language model doesn't have enough context about the annual report to create a well-informed response. For instance, the snapshot above has no mention of ‘2024’ in it so it would be ignored by traditional RAG approaches.

How to Implement Timestamped Embeddings in Python

Step 1: Adding Timestamps to Embeddings

You can append timestamps to text data during embedding creation. Here’s an example using Python’s datetime module.

from datetime import datetime
from sentence_transformers import SentenceTransformer

# Initialize embedding model
model = SentenceTransformer('all-MiniLM-L6-v2')

# Example text
text = "Asycd's blog performance metrics"

# Add timestamp
timestamp = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
text_with_timestamp = f"{text} [Timestamp: {timestamp}]"

# Generate embedding
embedding = model.encode(text_with_timestamp)
print("Generated Embedding:", embedding)

Step 2: Encoding the Timestamp with the Embedding

Depending on whether you are using third-party, vector index providers, you have the option to add metdata features such as date. This is where you plug in your datetime value for all relevant chunks you wish to upload to your vector index.

{
                "id": f"{filename}_chunk_{i}",
                "vector": embedding,
                "data": chunk,
                "metadata": {
                    "filename": filename,
                    "text": chunk,
                    "date": date_obj.strftime("%Y-%m-%d") if date_obj else None
                }
            }

Employing an AI Agent for Time-Specific Queries

A structured process for handling time-specific queries can further enhance accuracy. Consider this workflow:

1. Parse the query for temporal markers (e.g., “last month”).

2. Translate temporal markers into specific dates.

3. Search the vector database for embeddings within the date range.

4. Retrieve relevant chunks and generate a response.

Example Prompt:

You are a date extraction assistant. Extract date ranges from natural language queries.
                    Today's date is {current_date}.
                    
                    Return a JSON object with the following fields:
                    - date_from: Start date in YYYY-MM-DD format
                    - date_to: End date in YYYY-MM-DD format
                    - time_scope: The time unit (day, week, month, year, etc.)
                    - confidence: Your confidence in the extraction (0-1)
                    
                    Examples:
                    "Last month's sales" → {{"date_from": "2023-02-01", "date_to": "2023-02-28", "time_scope": "month", "confidence": 0.9}}
                    "Q1 performance" → {{"date_from": "2023-01-01", "date_to": "2023-03-31", "time_scope": "quarter", "confidence": 0.8}}
                    "2022 annual report" → {{"date_from": "2022-01-01", "date_to": "2022-12-31", "time_scope": "year", "confidence": 0.95}}
                    
                    If no date range is specified, use the current month.

Applications of Timestamped Embeddings

1. Financial Analysis
   Timestamping stock price data allows analysts to ask questions like “What was Apple’s stock price in August 2024?” without ambiguity.

2. Time-Aware Conversational AI
   Timestamped conversation history enables users to ask questions like “What did we discuss last week?” and receive accurate responses.

3. Dynamic Knowledge Bases
   In domains like news or technology, timestamped embeddings ensure that retrieved content reflects the latest developments.

Conclusion

Timestamped embeddings represent a significant advancement in Retrieval-Augmented Generation by integrating temporal awareness into AI systems. By leveraging Python’s datetime module and tools like vector databases, developers can build systems capable of delivering precise, time-contextual responses.

At Asycd, we strive to harness such innovations to empower our AI solutions. Whether you’re exploring timestamped embeddings or other cutting-edge techniques, Asycd provides insights and tools to help you stay at the forefront of AI development. Explore more about our work at Asycd today!