Athina AI

blogs

Semantic Caching For Faster, Smarter LLM Applications

Introduction In the rapidly evolving world of artificial intelligence, speed and accuracy are paramount. Enter semantic caching, a game-changing technology that's revolutionizing how AI applications handle data retrieval and processing. Unlike traditional caching methods, semantic caching brings a new level of intelligence to data storage and retrieval, making

blogs

Mixture of Experts: What You Really Need to Know in AI

Introduction In today’s time of rapid advancement in the evolution of artificial intelligence, myths can easily get to people. A very good example of that is the “Mixture of Experts” technique in large language models. As the name suggests, that would be a team of specialized models, but the

blogs

Building an Ideal Tech Stack for LLM Applications

Introduction As AI and machine learning continue to make rapid strides, developing LLM-powered applications has become both interesting and challenging. In this blog post, we will take you through all of the crucial components and tools that you'll need to build a robust tech stack from scratch for

blogs

Integrating Retrieval-Augmented Generation (RAG) in Your LLM Applications

Outline 1. Introduction * Importance and benefits of RAG 2. Understanding Retrieval-Augmented Generation * What is RAG? * Key differences from traditional LLMs 3. Steps to Integrate RAG * Choosing the right retrieval mechanism * Implementing the retrieval step * Integrating retrieval with generation 4. Best Practices * Optimizing for large-scale data * Fine-tuning RAG models * Avoiding common

blogs

Using Prompt Engineering to Control LLM Outputs

Outline 1. Introduction * Importance of controlling LLM outputs * Overview of prompt engineering 2. Understanding Prompt Engineering * What is prompt engineering? * Key principles of prompt engineering 3. Techniques for Effective Prompt Engineering * Using explicit instructions * Leveraging examples and few-shot learning * Controlling tone and style 4. Practical Examples * Case studies: From generic

blogs

Optimizing LLM Inference for Real-Time Applications

Outline 1. Introduction * Importance of optimizing LLM inference * Challenges in real-time applications 2. Understanding Inference Bottlenecks * Latency factors * Model size and complexity * Hardware limitations 3. Techniques for Optimizing Inference * Model quantization * Distillation and pruning * Batch processing and caching 4. Infrastructure and Deployment Considerations * Choosing the right hardware (GPUs, TPUs) * Edge

blogs

Supervised Fine-tuning (SFT) for LLMs

Introduction In the rapidly evolving world of artificial intelligence, large language models (LLMs) have become the cornerstone of numerous applications. But what happens when you need these models to perform specialized tasks? Enter the world of LLM fine-tuning – a powerful technique that bridges the gap between generic models and specific

research-papers

Training Language Models to Self-Correct via Reinforcement Learning

Original Paper: https://arxiv.org/abs/2409.12917 By: Aviral Kumar, Vincent Zhuang, Rishabh Agarwal, Yi Su, John D Co-Reyes, Avi Singh, Kate Baumli, Shariq Iqbal, Colton Bishop, Rebecca Roelofs, Lei M Zhang, Kay McKinney, Disha Shrivastava, Cosmin Paduraru, George Tucker, Doina Precup, Feryal Behbahani, Aleksandra Faust Abstract: Self-correction is

research-papers

Iteration of Thought: Leveraging Inner Dialogue for Autonomous Large Language Model Reasoning

Original Paper: https://arxiv.org/abs/2409.12618 By: Santosh Kumar Radha, Yasamin Nouri Jelyani, Ara Ghukasyan, Oktay Goktas Abstract: Iterative human engagement is a common and effective means of leveraging the advanced language processing power of large language models (LLMs). Using well-structured prompts in a conversational manner, human users

research-papers

To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoning

Original Paper: https://arxiv.org/abs/2409.12183 By: Zayne Sprague, Fangcong Yin, Juan Diego Rodriguez, Dongwei Jiang, Manya Wadhwa, Prasann Singhal, Xinyu Zhao, Xi Ye, Kyle Mahowald, Greg Durrett Abstract: Chain-of-thought (CoT) via prompting is the de facto method for eliciting reasoning capabilities from large language models (LLMs). But

research-papers

A Comprehensive Evaluation of Quantized Instruction-Tuned Large Language Models: An Experimental Analysis up to 405B

Original Paper: https://arxiv.org/abs/2409.11055 By: Jemin Lee, Sihyeong Park, Jinse Kwon, Jihun Oh, Yongin Kwon Abstract: Prior research works have evaluated quantized LLMs using limited metrics such as perplexity or a few basic knowledge tasks and old datasets. Additionally, recent large-scale models such as Llama 3.

research-papers

Large Language Monkeys: Scaling Inference Compute with Repeated Sampling

Original Paper: https://arxiv.org/abs/2407.21787 By: Bradley Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoc V. Le, Christopher Ré, Azalia Mirhoseini Abstract: Scaling the amount of compute used to train language models has dramatically improved their capabilities. However, when it comes to inference, we often limit the