Llama Cpp

Llama Cpp에 대해 3개 실제 데이터 소스에서 20개 공개 신호를 찾았습니다. GitHub 10건, Hacker News 0건, arXiv 10건을 원문 링크와 함께 보여줍니다.

관심도 점수

100

검색 slug
llama-cpp
실제 연결
Hacker News, GitHub, arXiv
키워드
and, the, models, with, for, llama, cpp, language
마지막 수집
2026-11-21T07:34:07.067Z

출처별 탭

The Geometry of Lies: Knowledge and Persona Directions for Detecting Deception in Large Language Models

Linear probes are the cheapest LLM lie detectors but often fail to transfer across kinds of lie. Label-free directions from honest and deceptive role-play are untested at 70B scale and on lies about knowledge, where a lie and an honest denial can be identical. We widen a published persona bank with roles that deny knowledge they have, plus honest counterparts. At each model's middle layer the resulting persona direction ranks lies across Liars' Bench better than single-dataset probes on three of its four models; i...

arxiv · 2026-11-19

Evaluating safeguarding in multi-turn conversations with AI assistants

Conversational AI assistants are now used for personal and emotional support. Developers report improvements in how new models respond to sensitive conversations, yet users and civil society continue to raise concerns about their behaviour in real-world, repeated interactions that safety evaluations rarely capture. Often, evaluations present signs of distress clearly and in a single message, even though people may reveal their struggles gradually in otherwise ordinary conversations. We worked with three community...

arxiv · 2026-11-19

BlockJack: Cross-Tenant Redirection Fault Attacks on Paged KV Cache Address Translation in LLM Serving

Large language model (LLM) serving frameworks accelerate inference by retaining previously computed states in a key-value (KV) cache. Paged serving frameworks such as vLLM organize this cache through block tables analogous to OS page tables, which map each request's logical KV blocks to physical KV blocks, forming a software-managed address translation layer. Although fault attacks such as Rowhammer have exploited OS page tables to redirect memory accesses, prior fault attacks on LLMs have largely targeted the dat...

arxiv · 2026-11-19

DTS: Delta-Based Tile Scoring for Efficient Long-Context Sparse Attention

Large language models (LLMs) increasingly need to process long documents and codebases, making long-context attention a major prefill bottleneck. Block sparse attention reduces computation by skipping unimportant blocks of the attention matrix, but scoring blocks accurately can itself be expensive. We propose Delta-Based Tile Scoring (DTS), a training-free, content-adaptive tile scoring method for sparse block attention. DTS selects query and key anchors according to local vector changes, using fewer representativ...

arxiv · 2026-11-19

Generative News Recommendation Reproduces Popularity: A Memorization-Based Analysis

Text-based generative recommenders can go beyond ranking existing items by directly generating item titles or other textual representations. In the news domain, this raises a societal concern that models may repeatedly surface already popular articles and thereby narrow the diversity of information to which users are exposed. We investigate training-data memorization as one mechanism behind this concentration. Using approximately 20 million browsing sessions from a Japanese news service, we continually trained an...

arxiv · 2026-11-19

RoCoKV: Accelerating LLM Inference Serving through Uncertainty and Redundancy Aware KV Cache Compression

KV cache compression is critical for large reasoning models, where most of the cost comes from long generated reasoning traces rather than the input prompt. These traces often contain redundant and low-utility tokens due to looping, self-verification, and corrections. However, such tokens may still receive high attention scores, making it difficult to remove them without harming the reasoning process. We propose RoCoKV, a KV cache compression method designed for large reasoning models. RoCoKV introduces two key co...

arxiv · 2026-11-19

Jailbreak Detection on Large Language Models via Contrastive Prototype Structures

Jailbreak attempts and harmful prompts remain a challenge for the safe deployment of large language models. We study whether contrastively learned representations can support safety detection without training an additional downstream classifier. We propose Jailbreak Detection with Prototype Inference (JDPI), which uses binary safety labels to construct harmful and benign prototypes and makes predictions from relative prototype distances. JDPI uses only binary safety labels for encoder adaptation, prototype constru...

arxiv · 2026-11-18

Chain-of-Thought Hidden States Reveal Task-Solving Directions for Direct Answers

Chain-of-Thought (CoT) prompting improves the performance of Large Language Models by eliciting intermediate reasoning steps. However, CoT is far from perfect: on some tasks, models perform worse than with direct-answer prompting, and on others they generate a few hundred tokens to reach the correct answer. We start by labelling each CoT hidden state as recovering if patching it into the direct prompt yields the correct answer, and failed otherwise. A linear probe shows that the model encodes these two classes dif...

arxiv · 2026-11-18

What the model knows but does not say: monitoring medical LLMs through verbalized thoughts

Efforts to keep clinical language models safe have concentrated on what enters and leaves the model, but a manipulated model can return an answer that looks entirely ordinary. Here we show that the model's own internal state, rendered in plain language, exposes manipulations the output conceals. Using a natural-language autoencoder, we translate the activations of a multimodal LLM (Gemma-3-27B) into text and let an input-blind judge read this verbalized internal state (the thoughts channel) and the model's answer...

arxiv · 2026-11-18

Test-Time Symmetrization: Scaling Laws for Model-Agnostic Learning under Structure

Test-time compute has emerged as a powerful way to improve the accuracy of language models at inference time, with scaling laws quantifying how accuracy improves with additional forward passes. Existing scaling laws, however, focus on repeated sampling from a fixed prompt, even though many tasks admit multiple equivalent representations of the same underlying problem. A graph does not depend on how its nodes are numbered, and a program does not depend on the names of its variables, yet language models can change t...

arxiv · 2026-11-18

AtomicBot-ai/atomic-agent

Atomic Agent is a local-first AI agent. Runs open-weight models on your own machine via llama.cpp.

github · 2026-04-21 · score 3,588 · comments 217 · stars 3,588 · forks 301

Michael-A-Kuykendall/shimmy

⚡ Pure-Rust WebGPU inference engine — OpenAI-API compatible, GGUF native, runs on any GPU. No Python. No llama.cpp. Single binary.

github · 2025-08-28 · score 6,021 · comments 9 · stars 6,021 · forks 587

ngxson/smolvlm-realtime-webcam

Real-time webcam demo with SmolVLM and llama.cpp server

github · 2025-05-12 · score 5,606 · comments 19 · stars 5,606 · forks 900

LearningCircuit/local-deep-research

~95% on SimpleQA (e.g. Qwen3.6-27B on a 3090). Supports all local and cloud LLMs (llama.cpp, Ollama, Google, ...). 10+ search engines - arXiv, PubMed, your private documents. Everything Local & Encrypted.

github · 2025-02-09 · score 9,618 · comments 1,620 · stars 9,618 · forks 941

h2oai/h2ogpt

Private chat with local GPT with document, images, video, etc. 100% private, Apache 2.0. Supports oLLaMa, Mixtral, llama.cpp, and more. Demo: https://gpt.h2o.ai/ https://gpt-docs.h2o.ai/

github · 2023-03-24 · score 11,913 · comments 328 · stars 11,913 · forks 1,294

abetlen/llama-cpp-python

Python bindings for llama.cpp

github · 2023-03-23 · score 10,692 · comments 725 · stars 10,692 · forks 1,489

serge-chat/serge

A web interface for chatting with Alpaca through llama.cpp. Fully dockerized, with an easy to use API.

github · 2023-03-19 · score 5,688 · comments 33 · stars 5,688 · forks 385

ggml-org/llama.cpp

LLM inference in C/C++

github · 2023-03-10 · score 135,888 · comments 2,607 · stars 135,888 · forks 25,832

PawanOsman/OpenCursor

Open-source Cursor-like AI coding agent for VS Code - agentic chat, multi-provider LLMs (OpenAI, Ollama, llama.cpp), semantic search, and MCP support

github · 2022-12-06 · score 6,047 · comments 10 · stars 6,047 · forks 1,034

intel/ipex-llm

Accelerate local LLM inference and finetuning (LLaMA, Mistral, ChatGLM, Qwen, DeepSeek, Mixtral, Gemma, Phi, MiniCPM, Qwen-VL, MiniCPM-V, etc.) on Intel XPU (e.g., local PC with iGPU and NPU, discrete GPU such as Arc, Flex and Max); seamlessly integrate with llama.cpp, Ollama, HuggingFace, LangChain, LlamaIndex, vLLM, DeepSpeed, Axolotl, etc.

github · 2016-08-29 · score 8,840 · comments 1,485 · stars 8,840 · forks 1,432

Honggi Developer Radar