This work presents a novel Voice in Head (ViH) framework, that integrates Large Language Models (LLMs) and the power of semantic understanding to enhance robotic navigation and interaction within complex environments. Our system strategically combines GPT and Gemini powered LLMs as Actor and Critic components within a reinforcement learning (RL) loop for continuous learning and adaptation. ViH employs a sophisticated semantic search mechanism powered by Azure AI Search, allowing users to interact with the system through natural language queries. To ensure safety and address potential LLM limitations, the system incorporates a Reinforcement Learning with Human Feedback (RLHF) component, triggered only when necessary. This hybrid approach delivers impressive results, achieving success rates of up to 94.54%, surpassing established benchmarks. Most importantly, the ViH framework offers a modular and scalable architecture. By simply modifying the environment, the system demonstrates the potential to adapt to diverse application domains. This research provides a significant advancement in the field of cognitive robotics, paving the way for intelligent autonomous systems capable of sophisticated reasoning and decision-making in real-world scenarios bringing us one step closer to achieving Artificial General Intelligence.