That’s best when possible, but if each context and response very large, a good prompt is the best way to distill the information without fine tuning.

I suppose you could try asking for shorter examples to provide for a few-shot prompt, could work well in some cases