my TRAMP connection (?error) to org file has duplicated my org entries

Question

now I have a file of a lot of such structure:

* one
:PROPERTIES:
:CUSTOM_ID: one
:END:
* one
:PROPERTIES:
:CUSTOM_ID: one
:END:
* one
:PROPERTIES:
:CUSTOM_ID: one
:END:
* one
:PROPERTIES:
:CUSTOM_ID: one
:END:
* two
BROWN FOX
* two
BROWN FOX
* two
BROWN FOX
* two
BROWN FOX

how can I revert it back so there is only one of every entity?

Answer 1

I have come up with a better-than-nothing solution: a python script

filename = ""
output_filename = ""


with open(filename, mode="r") as f:
    tmp = f.read()


blocks = tmp.split('\n*')


output_list = []
for block in blocks:
    if block not in output_list:
        output_list.append(block)

with open(output_filename, mode="w") as f:
    f.write("\n*".join(output_list))

I hope this helps someone