Small Rants on Evaluating LLMs with Function callsRight now I’m working on a project that extensively uses OpenAI API’s function calling capabilities (deprecated and called tools now).Jul 11Jul 11
Are you Cherry Picking Your Way to Prompt Engineering Success?Imagine this very common prompt engineering flow: You have a prompt engineering playing with a prompt, changing words, maybe in the OpenAI…Apr 26Apr 26
Case Study: Using Probability to Retry your way to LLM Output QualitySurprisingly for many practitioners of LLMs is the non deterministic nature of the LLM output. I’ve talked before about why this might be…Apr 23Apr 23
Looking into Non-Determinism in GPT-4I recently got into a lively discussion on an exploratory client call about LLM applications and where they are useful. The first thing I…Apr 20Apr 20
The Extraordinary Ineffectiveness of Prompt EngineeringPrompt Engineering + RAG has been all the rage. Even now, when I google “Prompt Engineering” I get over 250 million results with a heavy…Apr 20Apr 20
