Don't classify, hallucinate!

(softwaredoug.com)

74 points | by softwaredoug 3 days ago

16 comments

  • pu_pe 0 minutes ago
    Nice trick. Couldn't you embed the query though, compare it to the embedding of the categories, then ship only categories that are close to it in the prompt to a smaller model?
  • Majromax 31 minutes ago
    > In the notebook, I compute a MiniLM embedding of every real Wayfair classification. I compute the embedding of the fake, hypothetical embedding from the LLM. I then dot product the fake embedding into the real ones to find the most similar. Producing: [the right answer]

    Isn't this begging the question that the hallucinated classification will be more selective with respect to the real schema than the query itself? What would the dot product of <E(search query), E(schema)> have given?

    Even if that is too vague, smaller LLMs are capable rerankers; return the top N matching true categories and ask for a contextual ordering.

  • amitpoonia19xyz 12 minutes ago
    This is basically HyDE (Hypothetical Document Embeddings), no? I had tried this approach in the past, worked with limited success.
    • memjay 0 minutes ago
      We have this running in production. Can get pretty expensive and slow. We are trying to replace this with cheaper and faster methods that don’t hammer our LLM and elastic search endpoints as much.
  • ipsod 40 minutes ago
    Just this week I tried doing something similar with a nasty vibe-coded codebase I was trying to organize. I had Gemini Flash 3.6 classify each function/method in a similar way, giving a few plausible classifications for each (one agent per method).

    It didn't end up being very useful - I ran a comparison where I just had a bigger agent do the organization in a more straightforward way, and that had better results.

    I did find that Flash 3.6 High was >9x faster than Luna xhigh for this task, and got very similar results, though.

    • sheepscreek 24 minutes ago
      I’ve read a few different accounts, including OpenAI’s own admission, that Terra Medium or higher will likely produce better results than Luna xhigh and cost about the same or less.
  • piterrro 1 hour ago
    I would propose the following, query vector store for 10 closest categories based on a query, feed it to an LLM, in the prompt ask it to produce a single digit 0-9 representing the number of the most appropriate choice. Use plain text prompt, dont inflate token count with JSON. There you go, you just drastically reduced the output pricing.

    Additionally you could experiment with a reranker instead of an LLM or after reranking take top-3 results and then feed to LLM as input in order to reduce input token costs.

    • jddj 42 minutes ago
      Or press 9 to hear these options again
    • cesargstn 1 hour ago
      good this yeah
  • eka1 1 hour ago
    Did you validate this by running a A/B test? Main question is were you able to classify back into your known categories correctly all the time, or did the errors compound from the llm hallucination plus embedding search
    • softwaredoug 1 hour ago
      Using a Nano model, a tad worse than shipping a vocabulary to a larger OpenAI model. (And it’s an huge improvement on not classifying the queries at all).

      But no classification is perfect. In search in particular, you will also want to have places for manual intervention for high priority queries.

  • ashu1461 45 minutes ago
    Had stumbled on this library in the past

    https://github.com/aurelio-labs/semantic-router

    I guess it is based on the same fundamentals as well.

  • thatjoeoverthr 1 hour ago
    Smart! I've done the same trick for resolving extracted intents to selection.

    But if accuracy matters, you can't rely on embedding sort to get a closet match. With a real test set they usually don't hold up under scrutiny.

    Everything in AI is like this. You get an idea, try it once or twice, "LGTM" and you ship. Then it never survives contact reality.

    Embedding sort gives you a better shortlist than the whole list, but you will probably want a heavier model to vet candidates.

  • amelius 1 hour ago
    Can anyone explain why LLMs are so bad at finding products (their webpages) with given specifications?

    You'd think they would have solved it by now.

    • simonw 24 minutes ago
      LLMs aren't architected to handle filter-style comprehensive search without setting them up with additional tools.

      Asking an LLM for a list of every county in the USA for example, or every county with a population of more than 100,000 people.

      Even if those county names and their populations are mixed up in their weights, the nature of next-token-prediction does not lend them to effectively answering comprehensive, detailed questions like that.

      An agent system build on top of an LLM can do it, if it has access to tools which can help access eg a table of counties and then filter them with SQL or Pandas or similar.

      • amelius 13 minutes ago
        Yes, I was assuming they'd use external tools. Using only the raw LLM doesn't sound like a good strategy.

        Considering that agents are not a new concept, why isn't this a solved problem by now?

    • braiamp 58 minutes ago
      Because that's structured data and structured data is usually hidden away from users _and_ machines. Product rarely want to be honest, unless it's B2B in a very competitive market (and even then!). So, yeah, it's not that they are bad, it's that there are few good sources of information.

      (Lets ignore for now that no one seems to agree to what should be the spec sheets)

      • ashu1461 49 minutes ago
        With agentic commerce protocol / unified commerce protocol open ai and gemini are trying to solve this problem.

        The idea is to make structured queries using these protocols which can be used to fetch top products matching the user needs instead of just relying on semantic search.

        https://developers.openai.com/commerce/specs/file-upload/pro...

      • Zigurd 43 minutes ago
        It gets worse: shopping agents are hostile adversaries to Amazon unless they're paying Amazon and they've agreed to be friendly agents. No agent that won't betray you to an Amazon pricing strategy is going to be allowed access to Amazon structured data. They might even be fed poisoned data to discredit them.

        But you'll be amazed by the abundance.

      • amelius 50 minutes ago
        An LLM can read websites, right? And turn them into structured data.
        • ashu1461 47 minutes ago
          It can do that on run time, but it does not store data like that. The data is typically stored as embeddings in which it is hard to query data in a structured form. Example give me all products whose price is less than 200$ vs suggest me products for my spouse's birthday.
          • amelius 34 minutes ago
            Then they shouldn't store the data as embeddings.

            Instead: use an LLM to build a large (old-school) database of products with all their specifications. The LLM can also build the schema for that database as it finds more data.

            Then use an LLM to query that database based on the user's specifications (+ add some intelligence to find nice suggestions for a birthday if wanted, but I'd consider that an extra).

    • _flux 46 minutes ago
      Amazon Rufus has been mildly successful for me. I think the failures I've experienced with it are mostly because the product I'm looking for doesn't exist in the catalog.
      • pydry 42 minutes ago
        It's been an absolute fucking disaster for me. It hallucinates endlessly and its searches are terrible. It even managed to confidently gaslight me about there being a VAT invoice available for a specific product.

        I noticed yesterday when browsing on mobile that there used to be a box where I could search reviews and it got swapped with a Rufus box. I guess somebody needs to juice their engagement numbers for an investor briefing.

        honestly, Amazon doesnt even need AI it just needs a better UI, more metadata for its products and to make reviews less scammy.

        • sgc 27 minutes ago
          I asked a question once and now there is a effing alexa for shopping toolbar that takes a quarter of the screen that will not go away no matter how many times I close it, and the space remains taken even if I adblock it. Absolutely hostile implementation. I have words for this I cannot type out.
    • wslh 49 minutes ago
      Because the data, in general, is not included in the LLM model and it needs to search/browse for external information. It cannot look indefinitely so it get the top results from lists, not "evrything".
  • estetlinus 1 hour ago
    I was in a project where we sent the whole taxonomy every request, 40k tokens + one article, ”plz classify”. This was before structured outputs. It was extremely expensive and still hallucinated. Good ol’ days.
  • Colegno 1 hour ago
    Isn't search engines quicker than calling a LLM ? It might have a huge impact between a 20ms search engine call and a 2s LLM call for the end user.
    • quixoticaxolotl 54 minutes ago
      They are already solving the problem with search engines, they're just using an LLM as a first pass to create better embeddings to run a similarity match on first. The difference in latency is likely made up for in accuracy.
    • fastball 1 hour ago
      A 2s LLM call is pretty slow.
      • gadflyinyoureye 53 minutes ago
        Try using Digital Ocean. Minutes spent on inference.
    • nullsanity 52 minutes ago
      [dead]
  • sergiotapia 13 minutes ago
    This is a really great trick, woah!
  • apwheele 46 minutes ago
    This is another riff on not embedding a full document, but doing a summarization of the document and embedding the summary for RAG. Nice usecase for high cardinality data!
  • VladVladikoff 1 hour ago
    Eh, maybe you should keep both paths. When LLMs eventually crawl the site to feed back to agentic shoppers, maybe they logically follow the more truncated less decorated path.
  • einpoklum 19 minutes ago
    In the past, people would post advice on how to do something clever and useful yourself. Now, people post suggestions on how to talk out the side of their mouth to coax ther magic-8-ball slop generator to say something useful.
  • microgpt2 1 hour ago
    [dead]