2

Say I have two tests:

(defun func-one () nil)
(defun func-two () (and (func-one) t))

(ert-deftest test-one ()
  "Test basic functionality"
  (should (func-one)))

(ert-deftest test-two ()
  "Test advanced functionality"
  (should (func-two)))

How can I run test-two only if test-one succeeds?


Or, perhaps,

(defun func-one () nil)
(defun func-two () (and (func-one) t))

(ert-deftest test-all ()
  "Test basic functionality"
  (should (func-one))
  (should (func-two)))

Testing func-two only if func-one passes, for example.

Or even

(defun func-one () nil)
(defun func-two () (and (func-one) t))
(defun func-thr () (or  (func-one) t))

(ert-deftest test-one ()
  "Test basic functionality"
  (should (func-one)))

(ert-deftest test-two ()
  "Test advanced functionality"
  (should (func-two)))

(ert-deftest test-thr ()
  "Test advanced functionality"
  (should (func-thr)))

Testing

  • func-two only if func-one passes
  • func-thr only if func-one passes

but func-two failing should not impede func-thr from running.

6
  • Chain into a single test with and?
    – abo-abo
    Jan 1 '15 at 19:33
  • @abo-abo This will work for this simple case, but more complete testing suites will likely have a full-fledged DAG of tests. Jan 1 '15 at 19:34
  • @SeanAllred I'd argue that such a test suite is flawed. Test cases should be independent. What if the test you depend on is never run? If it's for resource management, create macros that handle this for you.
    – user227
    Jan 2 '15 at 12:32
  • @lunaryorn I agree that test cases should be independent, but in reality they aren't. I don't like having a 'failed' test when this failure is really the result of another function behaving improperly. The real use case is a function (A) that parses a string and a function (B) that uses the parsed information. Should (B) be marked 'failed' because (A) has failed? ('Yes' could very well be the right answer here -- I've never worked with CI in a professional setting.) Jan 2 '15 at 16:49
  • @SeanAllred In (B), don't call the function from (A) again. Instead create a literal value of whatever result (A) returns and use that in the test case for (B). Or keep things they are: It's perfectly ok if a single issue causes multiple tests to fail.
    – user227
    Jan 3 '15 at 11:26
2

I have used something like this in vc-tests.el:

(ert-deftest vc-test-git00-create-repo ()
  "Check `vc-create-repo' for the git backend."
  ...)

(ert-deftest vc-test-git01-register ()
  "Check `vc-register' and `vc-registered' for the git backend."
  (skip-unless
   (ert-test-passed-p
    (ert-test-most-recent-result
     (ert-get-test 'vc-test-git00-create-repo))))

  ...)
5
  • If I understand this correctly, this is almost exactly what I'm looking for! I'll try this out and get back to you :) Jan 2 '15 at 0:20
  • ERT doesn't guarantee any order of test execution, and the alphabetical ordering in current ERT solely depends on apropros-internal returning matching symbols in alphabetical order. Neither does it guarantee that all tests are run. This is pretty flaky, imho.
    – user227
    Jan 2 '15 at 12:30
  • @lunaryorn That's strange, then -- the form itself would suggest a dependency (ert-test-most-recent-result), but I suppose that's a farce :) ERT could certainly be patched to recognize this, though. It would actually be pretty easy if I'm not over-simplifying things in my head. Jan 2 '15 at 16:46
  • skip-unless is robust against errors. If the first test didn't run, the second one simply skips. Jan 2 '15 at 17:32
  • @lunaryorn maybe you file a "wishlist" bug towards Emacs asking for order of tests and alike? If time permits, I could check what to do. Jan 2 '15 at 17:34
1

Maybe something like this:

(defun func-one ()
  (message "running: func-one")
  nil)
(defun func-two ()
  (message "running: func-two")
  (and (func-one) t))
(defun func-thr ()
  (message "running: func three")
  (or  (func-one) t))

(require 'ert)
(ert-delete-all-tests)
(setq func-tests nil)
(ert-deftest test-one ()
  "Test basic functionality"
  (when (should (func-one))
    (push 'test-one func-tests)))

(ert-deftest test-two ()
  "Test advanced functionality"
  (should
   (and (memq 'test-one func-tests)
        (func-two))))

(ert-deftest test-thr ()
  "Test advanced functionality"
  (should
   (func-thr)))

(ert t)

I just put everything into one buffer and call eval-buffer to test this.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.