Bug Description
application/llm/anthropic.py uses the legacy Claude 2-era text completions API:
# lines 47-52
completion = self.anthropic.completions.create(
model=model,
max_tokens_to_sample=max_tokens,
prompt=f"{self.HUMAN_PROMPT} {prompt}{self.AI_PROMPT}",
)
This API is not supported by any Claude 3/3.5/4 model. Additionally both _raw_gen and _raw_gen_stream only extract messages[0] and messages[-1], silently discarding the entire multi-turn conversation history between them. All other LLM providers pass the full messages list.
Expected Behavior
AnthropicLLM should use client.messages.create() / client.messages.stream() (the modern Messages API), pass the full conversation history, and correctly handle system messages via the system= parameter.
Proposed Fix
- Remove deprecated
HUMAN_PROMPT/AI_PROMPT imports and instance variables
- Rewrite
_raw_gen to use anthropic.messages.create(), return response.content[0].text
- Rewrite
_raw_gen_stream to use anthropic.messages.stream(), yield text deltas
- Add
_split_messages() helper to separate system prompt from conversation turns
Bug Description
application/llm/anthropic.pyuses the legacy Claude 2-era text completions API:This API is not supported by any Claude 3/3.5/4 model. Additionally both
_raw_genand_raw_gen_streamonly extractmessages[0]andmessages[-1], silently discarding the entire multi-turn conversation history between them. All other LLM providers pass the fullmessageslist.Expected Behavior
AnthropicLLMshould useclient.messages.create()/client.messages.stream()(the modern Messages API), pass the full conversation history, and correctly handle system messages via thesystem=parameter.Proposed Fix
HUMAN_PROMPT/AI_PROMPTimports and instance variables_raw_gento useanthropic.messages.create(), returnresponse.content[0].text_raw_gen_streamto useanthropic.messages.stream(), yield text deltas_split_messages()helper to separate system prompt from conversation turns