Implications dataset into long words for fine tuning

Stream: Equational

[Michael Bucko]

A small experiment.
I turned the known implications into something like this. I'll fine tune a transformer, and see what will happen. Those are not words from the dictionary and they are long enough to not be treated as letters "x", "y", and so on.

The operation also has its name: special_operation_placeholder_treat_as_one

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa = aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa special_operation_placeholder_treat_as_one (bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb special_operation_placeholder_treat_as_one aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa)   aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa = aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa = aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa special_operation_placeholder_treat_as_one (bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb special_operation_placeholder_treat_as_one aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa)   aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa = aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa special_operation_placeholder_treat_as_one ((bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb special_operation_placeholder_treat_as_one cccccccccccccccccccccccccccccccccccccccccc) special_operation_placeholder_treat_as_one aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa)
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa = aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa special_operation_placeholder_treat_as_one (bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb special_operation_placeholder_treat_as_one aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa)   aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa = (aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa special_operation_placeholder_treat_as_one (bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb special_operation_placeholder_treat_as_one aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa)) special_operation_placeholder_treat_as_one (cccccccccccccccccccccccccccccccccccccccccc special_operation_placeholder_treat_as_one aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa)

[Michael Bucko]

For models that dont require chat formatted data. Our TSV into JSONL with prompt-completion pairs. For models like, for instance, O1-mini (a chat model), the data would still need to be chat-formatted data.

tsv2jsonl.py