All figures above are measured across six HAR benchmarks — USC-HAD, PAMAP2, MHEALTH, GOTOV, HHAR and Skoda — Token and latency numbers cover online inference only; the one-time offline design stage is reported separately below.
Human Activity Recognition (HAR) from wearable sensors supports applications in healthcare, rehabilitation, fitness tracking, and smart environments. Yet, existing deep learning approaches require dataset-specific training, large labeled corpora, and repeated adaptation to new sensor settings or activity taxonomies. Retrieval-Augmented Generation for Human Activity Recognition (RAG-HAR) addresses this by framing HAR as a training-free, retrieval-augmented task, in which statistical descriptions of sensor windows are used to retrieve similar labeled examples that guide LLM-based classification. We introduce RAG-HAR+, a retrieval-first and cost-optimized extension that strengthens retrieval while reducing dependence on LLM-based inference. RAG-HAR+ uses an offline Retrieval Designer Agent to design dataset-specific feature groups from a diverse pool of motion descriptors, enabling sensor windows to be compared using features better aligned with dataset-specific activity patterns. During inference, RAG-HAR+ uses majority voting over retrieved neighbors for samples with strong retrieval evidence and defers only uncertain cases to an LLM-based Ambiguity Resolver Agent. Across six HAR benchmarks, RAG-HAR+ maintains competitive or improved performance while reducing LLM usage, token consumption, and inference time. We further extend the RAG-HAR mobile prototype to demonstrate the practical feasibility of retrieval-first, LLM-assisted HAR in mobile sensing scenarios.
RAG-HAR+ keeps the training-free retrieval-augmented framework and changes when and where the LLM is used.
A Retrieval Designer Agent runs the LLM once per dataset: it designs three feature groups from a feature pool, builds a temporary index, scores it on a validation split by retrieval F1 and RAG hit rate, and refines over several rounds.
RAG-HAR described every window with the same basic statistics over fixed temporal segments. RAG-HAR+ selects dataset-specific feature groups, indexed as numerical vector fields for multi-vector search — so retrieval adapts to each dataset's motion characteristics.
Top-q neighbours are resolved by majority voting. A clear winner returns the activity with no LLM call at all; ties escalate to the Ambiguity Resolver Agent. Across the six test sets only 4.3% of samples needed the LLM.
Overview of RAG-HAR+. (a) Offline Retrieval Designer Agent for designing dataset-specific feature groups. (b) Feature indexing and inference pipeline, where training samples are indexed for retrieval and test samples are classified using majority voting or the Ambiguity Resolver Agent.
In RAG-HAR every test sample is sent to the LLM after retrieval, so online cost grows directly with the number of test samples. RAG-HAR+ invokes the Ambiguity Resolver Agent only when majority voting is undecided. Across the six benchmarks, online LLM usage falls from 220.8M tokens to 5.0M, a 97.7% reduction, and per-sample latency drops on every dataset.
(a) Average online LLM token usage. (b) Average inference latency.
| Dataset | Test samples | RAG-HAR tokens | RAG-HAR+ tokens | RAG-HAR+ LLM calls | Reduction |
|---|---|---|---|---|---|
| USC-HAD | 492 | 7,351,070 | 784,457 | 53 | 89.3% |
| PAMAP2 | 342 | 23,168,414 | 338,056 | 5 | 98.6% |
| MHEALTH | 666 | 35,368,862 | 52,847 | 1 | 99.9% |
| GOTOV | 1,324 | 35,984,861 | 1,490,004 | 53 | 95.9% |
| HHAR | 858 | 13,498,314 | 714,965 | 65 | 94.7% |
| Skoda | 793 | 105,415,235 | 1,655,811 | 14 | 98.4% |
| Total | 4,475 | 220,786,756 | 5,036,140 | 191 | 97.7% |
RAG-HAR issues one LLM call per test sample on every dataset. Reduction is computed from total tokens.
Latency. Speed-ups range from 5.7× on HHAR to 44.5× on MHEALTH, where only one test sample reaches the resolver (18.25 s → 0.41 s). RAG-HAR+ keeps per-sample latency between 0.41 s and 2.04 s across all datasets, which is compatible with continuous, near-real-time recognition on mobile and edge devices. The remaining latency comes from the retrieval pipeline itself, including feature extraction, normalization, and approximate nearest-neighbour search, rather than from network round trips.
Offline cost. The Retrieval Designer Agent uses 132,176 tokens in total across all six datasets (~$0.075), incurred once per dataset configuration and amortized across all later inference. Reported costs cover LLM API usage only.
RAG-HAR+ outperforms RAG-HAR on four of six datasets: USC-HAD, MHEALTH, HHAR and Skoda and remains competitive on PAMAP2 and GOTOV, while preserving the training-free retrieval-augmented design.
| Dataset | Best prior baseline | RAG-HAR | RAG-HAR+ | Δ |
|---|---|---|---|---|
| USC-HAD | 62.80 (Triplet LSTM) | 58.63 | 60.12 | +1.49 |
| PAMAP2 | 90.40 (Triplet LSTM) | 91.12 | 90.60 | −0.52 |
| MHEALTH | 96.47 (ADFE) | 96.74 | 98.19 | +1.45 |
| GOTOV | 79.40 (ALAE-TAE) | 79.92 | 75.97 | −3.95 |
| HHAR | 59.25 (Enhanced CPC) | 59.86 | 61.02 | +1.16 |
| Skoda | 94.80 (ALAE-TAE) | 95.21 | 97.74 | +2.53 |
Bold indicates the best F1 for that dataset. RAG-HAR+ achieves these results with 97.7% less online LLM usage.
For RAG-HAR, blue bars indicate retrieval F1 and red bars the final F1 after LLM classification. For RAG-HAR+, blue bars show the F1 achieved through retrieved-neighbour majority voting, and the stacked red segments the additional gain contributed by the Ambiguity Resolver Agent.
Key finding: In RAG-HAR+, retrieval alone achieves most of the final F1 score. The blue bars show the retrieval-only F1, obtained by majority voting over the retrieved neighbours before any LLM call. In RAG-HAR, this score was much lower than the final F1, so every sample had to be sent to the LLM. By contrast, the adaptive feature groups in RAG-HAR+ improve retrieval-only F1 by 3.8–19.0 points. For example, from 82.9% to 98.1% on MHEALTH, making retrieval reliable for most windows. The Ambiguity Resolver Agent then adds only the remaining 0.1–3.3 points, with the largest gains on USC-HAD and HHAR.
This stronger retrieval is still enabled by LLM reasoning, but that reasoning is moved to the offline feature-group design stage. The Retrieval Designer Agent uses the LLM once per dataset, reducing the need for repeated per-sample calls during inference. As a result, the cost no longer scales with the total number of test samples, but mainly with the number of ambiguous predictions.
@article{karunarathna2026ragharplus,
title={RAG-HAR+: Towards Cost-Efficient LLM-Based Human Activity Recognition for Edge Deployment},
author={Karunarathna, Hansi and Sivaroopan, Nirhoshan and Madarasingha, Chamara and Jayasumana, Anura and Thilakarathna, Kanchana},
journal={arXiv preprint arXiv:2607.26631},
year={2026}
}