How do I add appointments to effort sum?

Question

I have set up my appointments as below in my work.org.

* Work                                  :work:
** Client 1 Work
   <2020-06-03 Wed 09:00-10:00>
** Client 2 Work
   <2020-06-03 Wed 18:00-19:00>

And this is my org-agenda

Day-agenda (W23):
Wednesday   3 June 2020
               8:00...... ----------------
  work:        9:00-10:00 Client 1 Work                                                   :work::
              10:00...... ----------------
              10:14...... now - - - - - - - - - - - - - - - - - - - - - - - - -
              12:00...... ----------------
              14:00...... ----------------
              16:00...... ----------------
  work:       18:00-19:00 Client 2 Work                                                   :work::
              20:00...... ----------------

But when I go for my column view C-c C-x C-c in my agenda view, I notice that it my 1 hour appointments do not get added to my estimated effort or Actual effort.

I have setup the following in my .emacs

;; under custom-set-variables

'(org-agenda-time-grid
     (quote
      ((daily today require-timed remove-match)
       (800 1000 1200 1400 1600 1800 2000)
       "......" "----------------")))

'(org-columns-default-format
     "%25ITEM(Task) %TODO %TAGS %Effort(Estim){:}  %CLOCKSUM(ActTime)")

'(org-agenda-columns-add-appointments-to-effort-sum t)

I was hoping that setting org-agenda-columns-add-appointments-to-effort-sum as non-nil would help. The help shows the below

org-agenda-columns-add-appointments-to-effort-sum is a variable defined in ‘org-agenda.el’.
Its value is t
Original value was nil

Documentation:
Non-nil means the duration of an appointment will add to day effort.
The property to which appointment durations will be added is the one given
in the option ‘org-effort-property’.  If an appointment does not have
an end time, ‘org-agenda-default-appointment-duration’ will be used.  If that
is not set, an appointment without end time will not contribute to the time
estimate.

Quoting from the manual - https://orgmode.org/manual/Effort-Estimates.html

If you switch to column view in the daily/weekly agenda, the effort column summarizes the estimated work effort for each day81, and you can use this to find space in your schedule. To get an overview of the entire part of the day that is committed, you can set the option org-agenda-columns-add-appointments-to-effort-sum. The appointments on a day that take place over a specified time interval are then also added to the load estimate of the day.

My requirement is that my appointments get added up in some way to my column view in agenda mode, so that I know how much time (effort) is already allocated and thus will give me an idea how much more time I can allocate for myself during the day.

Org mode - 9.3.6 Emacs - 26.3

Answer 1

It seems that this is a bug in org-mode. Looking at the org-colview.el code I think that the reason why column view in org agenda can not add entry duration to efforts (even when org-agenda-columns-add-appointments-to-effort-sum is set to t) is because the function org-columns--collect-values which is called on each org entry by org-agenda-columns cannot actually get the text property called duration at the point where it tries to do that.

The fix below:

• treats property keys starting with AGENDA_* (e.g., AGENDA_TIME, AGENDA_DURATION) in your org-columns-default-format-for-agenda as special properties that will query text properties in agenda buffer instead of org properties
• if text properties in agenda buffer does not have duration value (AGENDA_DURATION key) it queries Effort property of the entry instead. This is the opposite behavior compared to what setting org-agenda-columns-add-appointments-to-effort-sum to t should do (i.e., duration overwrites the Effort instead of Effort overwrites duration)
• if the duration of the entry is negative (e.g. when you have a entry that spans the midnight <2020-08-03 Mon 23:00-7:00>) it adds 24h to the duration so time sums are calculated correctly

Add the following to your init.el:

(with-eval-after-load 'org-colview
    ;; adjusted org-agenda-columns function that now calls
    ;; org-agenda-columns--collect-values (see below) instead 
    ;; of org-columns--collect-values
    (defun org-agenda-columns ()
      "Turn on or update column view in the agenda."
      (interactive)
      (org-columns-remove-overlays)
      (if (markerp org-columns-begin-marker)
          (move-marker org-columns-begin-marker (point))
        (setq org-columns-begin-marker (point-marker)))
      (let* ((org-columns--time (float-time))
             (fmt
              (cond
               ((bound-and-true-p org-overriding-columns-format))
               ((bound-and-true-p org-local-columns-format))
               ((bound-and-true-p org-columns-default-format-for-agenda))
               ((let ((m (org-get-at-bol 'org-hd-marker)))
                  (and m
                       (or (org-entry-get m "COLUMNS" t)
                           (with-current-buffer (marker-buffer m)
                             org-columns-default-format)))))
               ((and (local-variable-p 'org-columns-current-fmt)
                     org-columns-current-fmt))
               ((let ((m (next-single-property-change (point-min) 'org-hd-marker)))
                  (and m
                       (let ((m (get-text-property m 'org-hd-marker)))
                         (or (org-entry-get m "COLUMNS" t)
                             (with-current-buffer (marker-buffer m)
                               org-columns-default-format))))))
               (t org-columns-default-format)))
             (compiled-fmt (org-columns-compile-format fmt)))
        (setq org-columns-current-fmt fmt)
        (when org-agenda-columns-compute-summary-properties
          (org-agenda-colview-compute org-columns-current-fmt-compiled))
        (save-excursion
          ;; Collect properties for each headline in current view.
          (goto-char (point-min))
          (let (cache)
            (while (not (eobp))
              (let ((m (org-get-at-bol 'org-hd-marker)))
                (when m
                  (push (cons (line-beginning-position)
                              ;; `org-columns-current-fmt-compiled' is
                              ;; initialized but only set locally to the
                              ;; agenda buffer.  Since current buffer is
                              ;; changing, we need to force the original
                              ;; compiled-fmt there.
                              ;; clavis
                              (org-agenda-columns--collect-values compiled-fmt m))
                        cache)))
              (forward-line))
            (when cache
              (org-columns--set-widths cache)
              (org-columns--display-here-title)
              (when (setq-local org-columns-flyspell-was-active
                                (bound-and-true-p flyspell-mode))
                (flyspell-mode 0))
              (dolist (entry cache)
                (goto-char (car entry))
                (org-columns--display-here (cdr entry)))
              (setq-local org-agenda-columns-active t)
              (when org-agenda-columns-show-summaries
                (org-agenda-colview-summarize cache)))))))

  ;; new function that substitutes org-columns--collect-values
  (defun org-agenda-columns--collect-values (&optional compiled-fmt m)
      "Collect values for columns on the current line.

  Return a list of triplets (SPEC VALUE DISPLAYED) suitable for
  `org-columns--display-here'.

  This function assumes `org-columns-current-fmt-compiled' is
  initialized is set in the current buffer.  However, it is
  possible to override it with optional argument COMPILED-FMT."
      (let ((summaries (get-text-property (point) 'org-summaries)))
        (mapcar
         (lambda (spec)
           (pcase spec
             (`(,p . ,_)
              (let* ((v (or (cdr (assoc spec summaries))
                            ;; first check if p is a special agenda (text) property
                            (when-let* ((prop-p (string-match "^AGENDA_\\(.+\\)" p))
                                        (prop-name (downcase (match-string 1 p))))
                              ;; if property is duration consider effort if it is not set
                              (if (string= prop-name "duration")
                                  (if-let ((dur (org-get-at-bol (intern prop-name))))
                                      ;; if duration is negatie (i.e. time range crossess
                                      ;; midnight) then add 24h to duration
                                      (if (> dur 0.0)
                                          (propertize (org-duration-from-minutes dur)
                                                      'face 'org-scheduled)
                                        (propertize (org-duration-from-minutes
                                                     (+ dur (* 24 60)))
                                                    'face 'org-scheduled))
                                    (org-with-point-at m
                                      (org-entry-get
                                       (point) org-effort-property
                                       'selective t)))
                                (org-get-at-bol (intern prop-name))))
                            (org-with-point-at m
                              (org-entry-get (point) p 'selective t))
                            "")))
                ;; A non-nil COMPILED-FMT means we're calling from Org
                ;; Agenda mode, where we do not want leading stars for
                ;; ITEM.  Hence the optional argument for
                ;; `org-columns--displayed-value'.
                (list spec v (org-columns--displayed-value spec v compiled-fmt))))))
         (or compiled-fmt org-columns-current-fmt-compiled)))))

Adding the following setup in org-agenda-custom-commands

(add-to-list 'org-agenda-custom-commands
             '("j" "Planning Table"
               agenda ""
               ((org-agenda-overriding-header "")
                (org-agenda-span 1)
                (org-agenda-use-time-grid nil)
                (org-agenda-view-columns-initially t)
                (org-columns-default-format-for-agenda
                 "%11AGENDA_TIME(When) %4TODO(Type) %40ITEM(What) %5AGENDA_DURATION(Takes){:}")
                ;; do not show wardings, overdue and overscheduled
                (org-scheduled-past-days 0)
                (org-deadline-past-days 0)
                (org-deadline-warning-days 0)
                ;; skip finished entries
                (org-agenda-skip-deadline-if-done t)
                (org-agenda-skip-scheduled-if-done t))))

and with this org file

* TODO Buy last minute spaceship tickets
:PROPERTIES:
:Effort:   1:00
:END:
<2020-08-03 Mon>


* TODO Take up space suit from dry cleaning
:PROPERTIES:
:Effort:   1:00
:END:
<2020-08-03 Mon 14:00-14:30>

* Rest well before space flight
<2020-08-03 Mon 22:00-06:00>

* Taxi to space port
<2020-08-04 Tue 07:00-07:30>

* Trip to Mars
<2020-08-04 Tue 08:30>--<2021-01-13 Wed>

You should be able to get the following agenda:

Hope this will help for now. Please, mind that this is not a sustainable solution as it will likely break if new versions of org will change org-columns code.

Org 9.3.7. Emacs 26.3