5

I'm trying to write a piece of code that runs a single ERT test programmatically. In the ERT manual, they show an example that creates a test body and then runs the body, like so:

(ert-deftest ert-test-record-backtrace ()
   (let ((test (make-ert-test :body (lambda () (ert-fail "foo")))))
    (let ((result (ert-run-test test)))

But... (ert-run-test won't work with a normal ert-deftest name such as the one shown above, ert-test-record-backtrace. If I try doing that I see an error

(ert-run-test 'ert-test-record-backtrace)
Debugger entered--Lisp error: (wrong-type-argument arrayp my-deftest)

The code (ert-run-tests-interactively) runs the usual ERT function that prompts for tests in the minibuffer.

How can I run one or more ERT tests programmatically? Is it even possible? I'm working on this kind of syntax right now, but it is not working. I think I need to extract the function body value of my-deftest, but I don't know how to do that yet.

(ert-run-test (make-ert-test :body my-deftest))

Update: I tried to put my whole test function inside the make-ert-test format shown in the example above, and was able to run the test. Not pretty code, because of the indenting. And the result was in square brackets, which I don't understand yet. But I can see the value: t in there that I would like to extract. So I'm getting closer.

(setq result
  (ert-run-test
   (make-ert-test :body
                  (lambda ()
                    ;; my ert test code here
                    (should (equal tcoll output))
                    ))))

[cl-struct-ert-test-passed "" (((should ...) :form (equal ... ...) :value t :explanation nil))]

After more experimentation to dig the answer out of the result returned (it was an array), here is the code that gets me to the t that I could see.

(setq data (aref result 2))
(((should (equal tcoll output)) :form (equal ...)) :value t :explanation nil))
(nth 4 (car data))
t

It works, but it seems like a brute force approach because I have to copy my ert-deftest code into a separate lambda function as shown above.

3

Let's use the example from the ERT manual:

(ert-deftest ert-test-record-backtrace ()
  (let ((test (make-ert-test :body (lambda () (ert-pass "foo")))))
    (let ((result (ert-run-test test)))
      (should (ert-test-failed-p result)))))

The test should pass obviously. A list of symbols naming an ert-test object will be returned with

(apropos-internal "" #'ert-test-boundp) ;⇒ (ert-test-record-backtrace)

And finally show the number of available tests that passed as expected:

(ert--stats-passed-expected (ert-run-tests 't (lambda (&rest args)))) ;⇒ 1

The listener in ert-run-tests may be an arbitrary function called for side-effects.

  • Thank you for your help. With apropos-internal I can see my deftest names, but they are not test objects, since ert-run-test rejects them with wrong type argument errors. I will go an experiment with listeners now, since I've never used them either. – Kevin Jun 22 '16 at 16:29
  • (ert-test-passed-p (ert-run-test (ert-get-test 'ert-test-record-backtrace))) is what you're looking for. – mutbuerger Jun 22 '16 at 16:35
  • The (ert--stats-passed-expected (ert-run-tests 'my-deftest ..listener)) syntax was closest to answering my question, I think. I can work with it. But I find it quite odd that in order to do something "so simple" as running a single test programmatically, I must call one of the "don't call functions with double hyphens in the name" functions. Thank you again. – Kevin Jun 22 '16 at 16:37
  • 1
    Sorry, our comments were out of sync. You are exactly right, (ert-test-passed-p ...) was exactly what I was looking for. Thank you again. I wonder why I couldn't find any of this in the manuals or online doc... Did I miss something obvious, or did you find out this info by working with the source code or something? – Kevin Jun 22 '16 at 17:54
  • You're welcome. IIRC the ert-modeline package got me inspired to look into ERT. In addition, source code is always enlightening. – mutbuerger Jun 22 '16 at 18:03
3

You can try ert-run-tests-batch, e.g.

(ert-run-tests-batch 'ert-test-record-backtrace)

but the results are not returned as a value, but rather via messages in *Messages* (or on stdout). Otherwise, you can use ert-run-tests which is a bit more complex to use because you need to provide a listener function:

(ert-run-tests 'ert-test-record-backtrace
               (lambda (event &rest args)
                 (message "Test event %S: %S" event args)))
  • Thank you for your test event example. Wow, what a huge amount of output from the little listener that dumps the event and args (and my little deftest function was tiny!). – Kevin Jun 22 '16 at 16:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.