2

I just learned about the following org-mode option, which ensures newly entered timestamps end up in the LOGBOOK drawer:

(setq org-log-into-drawer t)

How can I automatically move existing timestamps (which so far have been logged outside of the drawer) so that they all end up inside the drawer?

As an example, how to quickly go from:

** TODO Important task
SCHEDULED: <2018-04-23 Mon .+7d>
- State "DONE"       from "TODO"       [2018-04-09 Mon 14:05]
CLOCK: [2018-04-09 Mon 14:02]--[2018-04-09 Mon 14:05] =>  0:03
- State "DONE"       from "TODO"       [2018-03-26 Mon 09:21]
- State "DONE"       from "TODO"       [2018-03-19 Mon 13:15]
CLOCK: [2018-03-19 Mon 13:09]--[2018-03-19 Mon 13:15] =>  0:06
- State "DONE"       from "TODO"       [2018-03-01 Thu 12:39]
CLOCK: [2018-02-26 Mon 08:43]--[2018-02-26 Mon 08:56] =>  0:13
- State "DONE"       from "TODO"       [2018-01-18 Thu 08:20]
- State "DONE"       from "TODO"       [2017-11-13 mån 14:07]
CLOCK: [2017-11-13 mån 14:06]--[2017-11-13 mån 14:07] =>  0:01
- State "DONE"       from "TODO"       [2017-10-23 mån 17:55]
CLOCK: [2017-10-23 mån 10:08]--[2017-10-23 mån 10:11] =>  0:03
- State "DONE"       from "TODO"       [2017-10-02 mån 10:46]
CLOCK: [2017-10-02 mån 10:44]--[2017-10-02 mån 10:46] =>  0:02

To something like:

** TODO Important task
SCHEDULED: <2018-04-23 Mon .+7d>
:LOGBOOK:
CLOCK: [2018-04-09 Mon 14:02]--[2018-04-09 Mon 14:05] =>  0:03
CLOCK: [2018-03-19 Mon 13:09]--[2018-03-19 Mon 13:15] =>  0:06
CLOCK: [2018-02-26 Mon 08:43]--[2018-02-26 Mon 08:56] =>  0:13
CLOCK: [2017-11-13 mån 14:06]--[2017-11-13 mån 14:07] =>  0:01
CLOCK: [2017-10-23 mån 10:08]--[2017-10-23 mån 10:11] =>  0:03
CLOCK: [2017-10-02 mån 10:44]--[2017-10-02 mån 10:46] =>  0:02
:END:
- State "DONE"       from "TODO"       [2017-10-23 mån 17:55]
- State "DONE"       from "TODO"       [2017-10-02 mån 10:46]
[...]
  • Please provide a specific example of where your time-stamp(s) is/are that you want to move, and which drawer you wish to target (e.g., LOGBOOK), and what should happen if there are two times-tamps; e.g., DEADLINE and SCHEDULED. Some people put time-stamps in a heading, and some people put time-stamps in the notes section. So, as you see, including the drawer, there are at least five possible locations. – lawlist Apr 19 '18 at 15:23
  • Good point. I updated my question. – Roberto Aloi Apr 20 '18 at 6:37
  • I had a similar problem and found two answers. If the native solution fails, can you check the value of the variable org-clock-into-drawer (C-h v org-clock-into-drawer, mine is t) and the return value of the function (org-clock-into-drawer) (M-: (org-clock-into-drawer), mine is LOGBOOK)? – miguelmorin Sep 24 at 20:36
1

Native solution

From the documentation of org-clock-into-drawer (see it with C-h f org-clock-into-drawer):

Non-nil when clocking info should be wrapped into a drawer.

When non-nil, clocking info will be inserted into the same drawer
as log notes (see variable ‘org-log-into-drawer’), if it exists,
or "LOGBOOK" otherwise.  If necessary, the drawer will be
created.

When an integer, the drawer is created only when the number of
clocking entries in an item reaches or exceeds this value.

When a string, it becomes the name of the drawer, ignoring the
log notes drawer altogether.

Therefore, after running (setq org-clock-into-drawer t) and clocking in on a headline, any lines under that headline that are a clock segment such as:

CLOCK: [2018-04-09 Mon 14:02]--[2018-04-09 Mon 14:05] =>  0:03

are first moved into a drawer, creating one if necessary and in reverse order that they appear, and the new clock time is added at the top.

Solution in Python

I disliked the reverse order and am too unfamiliar with Emacs LISP to adapt the source code from function org-clock-find-position (lines 1508 onwards of org-clock.el.gz in my version). So I used Python 3.

The code below runs the doc-tests first and only if they all pass does it process the file. It also moves into the drawer any lines that are state changes:

- State "DONE"       from "TODO"       [2019-07-23 Tue 23:11]

It keeps the order of appearance of lines that are clock segments or state changes. It creates a backup of the file before running it. I inspected the result manually and could not find any errors.

Save as <name-of-script>, then call with python3 <name-of-script> -f <path-to-org-file>.

import argparse
import doctest
import os
import re
import shutil

DEFAULT_FILE = os.path.expanduser("~/org/gtd.org")
LOGBOOK_START = ":LOGBOOK:"
LOGBOOK_END = ":END:"

META_DATA_RE = "^( |\\t)*:?(CLOSED|DEADLINE|SCHEDULED|END|#+END):"
HEADLINE_RE = "[*]+ "

def find_end_of_metadata(lines):
    """Finds the last index of metadata in the array
    >>> lines = ['** TODO task', \
    'SCHEDULED: <2019-09-24 Tue 08:04 ++1d>', \
    ':PROPERTIES:', \
    ':LAST_REPEAT: [2019-09-23 Mon 11:42]', \
    ':END:', \
    ':LOGBOOK:', \
    '...', \
    ':END:', \
    'some comment']
    >>> find_end_of_metadata(lines) == len(lines) - 2
    True
    """

    if list != type(lines):
        raise ValueError("This function requires a list as input")

    num_lines = len(lines)
    for i in range(num_lines - 1, -1, -1):
        line = lines[i]
        if re.match(META_DATA_RE, line) or re.match(HEADLINE_RE, line):
            return i
    assert False, "Unable to find end of metadata!" + "\n".join(lines)

def process_headline(headline, logbook_start = LOGBOOK_START, logbook_end = LOGBOOK_END):
    """Processes the headline, creating a logbook drawer if needed, and
    moving state changes and clocks into the logbook drawer.
    >>> no_drawer_headline = '\\n'.join(['** TODO task', \
    'SCHEDULED: <2019-09-19 Thu 21:00 ++1d>', \
    ':PROPERTIES:', \
    ':LAST_REPEAT: [2019-09-18 Wed 21:24]', \
    ':END:', \
    '- State "DONE"       from "TODO"       [2019-09-18 Wed 21:24]', \
    '- State "DONE"       from "TODO"       [2019-09-17 Tue 12:52]', \
    'CLOCK: [2019-09-24 Tue 09:01]--[2019-09-24 Tue 09:41] =>  0:40', \
    'CLOCK: [2019-04-22 Mon 14:31]--[2019-04-22 Mon 15:23] =>  0:52'])
    >>> no_drawer_actual = process_headline(no_drawer_headline)
    >>> no_drawer_expected = '\\n'.join(['** TODO task', \
    'SCHEDULED: <2019-09-19 Thu 21:00 ++1d>', \
    ':PROPERTIES:', \
    ':LAST_REPEAT: [2019-09-18 Wed 21:24]', \
    ':END:', \
    ':LOGBOOK:', \
    '- State "DONE"       from "TODO"       [2019-09-18 Wed 21:24]', \
    '- State "DONE"       from "TODO"       [2019-09-17 Tue 12:52]', \
    'CLOCK: [2019-09-24 Tue 09:01]--[2019-09-24 Tue 09:41] =>  0:40', \
    'CLOCK: [2019-04-22 Mon 14:31]--[2019-04-22 Mon 15:23] =>  0:52', \
    ':END:'])
    >>> drawer_headline = '\\n'.join(['** TODO task', \
    'SCHEDULED: <2019-09-19 Thu 21:00 ++1d>', \
    ':PROPERTIES:', \
    ':LAST_REPEAT: [2019-09-18 Wed 21:24]', \
    ':END:', \
    ':LOGBOOK:', \
    '- State "DONE"       from "TODO"       [2019-09-18 Wed 21:24]', \
    'CLOCK: [2019-09-24 Tue 09:01]--[2019-09-24 Tue 09:41] =>  0:40', \
    ':END:', \
    '- State "DONE"       from "TODO"       [2019-09-17 Tue 12:52]', \
    'CLOCK: [2019-04-22 Mon 14:31]--[2019-04-22 Mon 15:23] =>  0:52'])
    >>> drawer_actual = process_headline(drawer_headline)
    >>> drawer_expected = '\\n'.join(['** TODO task', \
    'SCHEDULED: <2019-09-19 Thu 21:00 ++1d>', \
    ':PROPERTIES:', \
    ':LAST_REPEAT: [2019-09-18 Wed 21:24]', \
    ':END:', \
    ':LOGBOOK:', \
    '- State "DONE"       from "TODO"       [2019-09-18 Wed 21:24]', \
    'CLOCK: [2019-09-24 Tue 09:01]--[2019-09-24 Tue 09:41] =>  0:40', \
    '- State "DONE"       from "TODO"       [2019-09-17 Tue 12:52]', \
    'CLOCK: [2019-04-22 Mon 14:31]--[2019-04-22 Mon 15:23] =>  0:52', \
    ':END:'])
    >>> no_drawer_actual == no_drawer_expected
    True
    >>> drawer_actual == drawer_expected
    True
    """

    # Split by lines
    lines = re.split("\n|\r", headline)

    # Find the indices of all the lines that are a state change
    # and copy them over
    indices = []
    logbook_lines = []
    for iLine, line in enumerate(lines):
        if line.startswith('- State') or line.startswith('CLOCK: '):
            indices.append(iLine)
            logbook_lines.append(line)

    if 0 == len(logbook_lines):
        return headline

    # Delete lines from original
    for iLine in reversed(indices):
        lines.pop(iLine)

    # Initialize new array to hold result
    processed_lines = []

    # Find index of logbook drawer, if any
    if logbook_start in lines:
        logbook_start_index = lines.index(logbook_start)
        logbook_end_index = lines.index(logbook_end, logbook_start_index)
    else:
        logbook_start_index = find_end_of_metadata(lines) + 1
        lines = lines[:logbook_start_index] + [logbook_start, logbook_end] + lines[logbook_start_index:]
        logbook_end_index = logbook_start_index + 1

    # Add clock lines in the logbook drawer
    return "\n".join(lines[:(logbook_start_index + 1)] + logbook_lines + lines[logbook_end_index:])

def split_headlines(s):
    """Splits the contents of an Org file by headline and keeps the delimiter that marks it.
    >>> contents = "\\n".join(['* Level 1', \
    '** TODO Level 2', \
    'comments', \
    '** Level 2'])
    >>> actual = split_headlines(contents)
    >>> expected = ['* Level 1', \
    '\\n'.join(['** TODO Level 2', 'comments']),
    '** Level 2']
    >>> actual == expected
    True
    """

    regex = re.compile("(\\n|\\r)" + HEADLINE_RE)
    matches = []
    prev_end = 0
    for match in regex.finditer(s):
        match_start = match.start()
        matches.append(s[prev_end:match.start()])

        prev_end = match_start
        if s[prev_end] in ['\n', '\r']:
            prev_end += 1

    if prev_end < len(s):
        matches.append(s[prev_end:])

    return matches

def process_file(filepath):
    """For the org file in the argument, moves the state changes and clock segments into drawers.
    """

    # Make a backup, just in case
    backup = filepath + "_backup"
    assert not os.path.exists(backup), "Backup file already exists!"
    shutil.copy(filepath, backup)
    assert os.path.exists(backup)

    with open(filepath, "r") as f:
        contents = f.read(-1)

    with open(backup, "r") as f:
        backup_contents = f.read(-1)

    assert contents == backup_contents

    # Split by headlines
    headlines = split_headlines(contents)

    # Process each
    new_headlines = []
    count = 0
    for h in headlines:
        new_h = process_headline(h)
        new_headlines.append(new_h)

    # Write to file
    with open(filepath, "w") as f:
        f.write("\n".join(new_headlines))

def get_args():
    parser = argparse.ArgumentParser(description = "Process text in an Org file")
    parser.add_argument("-f", "--file", default=DEFAULT_FILE)

    return parser.parse_args()


def main():

    args = get_args()
    filepath = args.file

    process_file(filepath)

if "__main__" == __name__:
    doctests = doctest.testmod()
    assert 0 == doctests.failed, "Some doc-tests failed, will not run the script"
    main()

Cross-posted from this other thread that deals with lines with state changes.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.