# GPT-5.4

This work presents the first user study assessing the reliability of LLM-as-a-judge for evaluating conversational recommendation system (CRS) responses: 20 multi-turn sessions were sampled from TalkPlayData-Challenge, candidate responses were generated by four instruction-tuned LLMs, and 20 music-domain experts produced n=400 ratings on Personalization Quality and Explanation Quality; bootstrapped correlation analysis over 10,000 iterations found moderate positive alignment between LLM judges and human assessments (highest r=0.55 for Gemini-3.1Pro on personalization), outperforming all reference-based baselines.

This work presents the first user study assessing the reliability of LLM-as-a-judge for evaluating conversational recommendation system (CRS) responses: 20 multi-turn sessions were sampled from TalkPlayData-Challenge, candidate responses were generated by four instruction-tuned LLMs, and 20 music-domain experts produced n=400 ratings on Personalization Quality and Explanation Quality; bootstrapped correlation analysis over 10,000 iterations found moderate positive alignment between LLM judges and human assessments (highest r=0.55 for Gemini-3.1Pro on personalization), outperforming all reference-based baselines.

This study investigates medical concept normalization of short German clinical expressions to SNOMED CT by comparing a direct GPT-5.4 LLM-only approach with a hybrid approach combining medBERT.de bi-encoder embedding retrieval and RAG-based LLM reranking, finding that the LLM-only baseline achieves Recall@1 of 0.235, Recall@3 of 0.297, and Recall@5 of 0.303, while embedding-based retrieval reaches Recall@1 of 0.681, Recall@3 of 0.783, and Recall@5 of 0.812, and adding RAG reranking further improves Recall@1 to 0.771 with Recall@3 and Recall@5 at 0.809 and 0.812.

This study investigates medical concept normalization of short German clinical expressions to SNOMED CT by comparing a direct GPT-5.4 LLM-only approach with a hybrid approach combining medBERT.de bi-encoder embedding retrieval and RAG-based LLM reranking, finding that the LLM-only baseline achieves Recall@1 of 0.235, Recall@3 of 0.297, and Recall@5 of 0.303, while embedding-based retrieval reaches Recall@1 of 0.681, Recall@3 of 0.783, and Recall@5 of 0.812, and adding RAG reranking further improves Recall@1 to 0.771 with Recall@3 and Recall@5 at 0.809 and 0.812.