1

Consider the following snippet

(defun python-shell-filter-long-lines (string)
  (print string)
  string)

(push 'python-shell-filter-long-lines comint-preoutput-filter-functions)

and M-x run-python then input str(list(range(1000))), the output of in the comint buffer is a very long line containing '[0, 1, ..., 999]', while the*Message* buffer gives 

"'[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207, 208, 209, 210, 211, 212, 213, 214, 215, 216, 217, 218, 219, 220, 221, 222, 223, 224, 225, 22"

"6, 227, 228, 229, 230, 231, 232, 233, 234, 235, 236, 237, 238, 239, 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260, 261, 262, 263, 264, 265, 266, 267, 268, 269, 270, 271, 272, 273, 274, 275, 276, 277, 278, 279, 280, 281, 282, 283, 284, 285, 286, 287, 288, 289, 290, 291, 292, 293, 294, 295, 296, 297, 298, 299, 300, 301, 302, 303, 304, 305, 306, 307, 308, 309, 310, 311, 312, 313, 314, 315, 316, 317, 318, 319, 320, 321, 322, 323, 324, 325, 326, 327, 328, 329, 330, 331, 332, 333, 334, 335, 336, 337, 338, 339, 340, 341, 342, 343, 344, 345, 346, 347, 348, 349, 350, 351, 352, 353, 354, 355, 356, 357, 358, 359, 360, 361, 362, 363, 364, 365, 366, 367, 368, 369, 370, 371, 372, 373, 374, 375, 376, 377, 378, 379, 380, 381, 382, 383, 384, 385, 386, 387, 388, 389, 390, 391, 392, 393, 394, 395, 396, 397, 398, 399, 400, 401, 402, 403, 404, 405, 406, 407, 408, 409, 410, 411, 412, 413, 414, 415, 416, 417, 418, 419, 420, 421, 422, 423, 424, 425, 426, 427, 428, 429, 430, 4"

"31, 432, 433, 434, 435, 436, 437, 438, 439, 440, 441, 442, 443, 444, 445, 446, 447, 448, 449, 450, 451, 452, 453, 454, 455, 456, 457, 458, 459, 460, 461, 462, 463, 464, 465, 466, 467, 468, 469, 470, 471, 472, 473, 474, 475, 476, 477, 478, 479, 480, 481, 482, 483, 484, 485, 486, 487, 488, 489, 490, 491, 492, 493, 494, 495, 496, 497, 498, 499, 500, 501, 502, 503, 504, 505, 506, 507, 508, 509, 510, 511, 512, 513, 514, 515, 516, 517, 518, 519, 520, 521, 522, 523, 524, 525, 526, 527, 528, 529, 530, 531, 532, 533, 534, 535, 536, 537, 538, 539, 540, 541, 542, 543, 544, 545, 546, 547, 548, 549, 550, 551, 552, 553, 554, 555, 556, 557, 558, 559, 560, 561, 562, 563, 564, 565, 566, 567, 568, 569, 570, 571, 572, 573, 574, 575, 576, 577, 578, 579, 580, 581, 582, 583, 584, 585, 586, 587, 588, 589, 590, 591, 592, 593, 594, 595, 596, 597, 598, 599, 600, 601, 602, 603, 604, 605, 606, 607, 608, 609, 610, 611, 612, 613, 614, 615, 616, 617, 618, 619, 620, 621, 622, 623, 624, 625, 626, 627, 628, 629, 630, 631, 632, 633, 634, 635, "

"636, 637, 638, 639, 640, 641, 642, 643, 644, 645, 646, 647, 648, 649, 650, 651, 652, 653, 654, 655, 656, 657, 658, 659, 660, 661, 662, 663, 664, 665, 666, 667, 668, 669, 670, 671, 672, 673, 674, 675, 676, 677, 678, 679, 680, 681, 682, 683, 684, 685, 686, 687, 688, 689, 690, 691, 692, 693, 694, 695, 696, 697, 698, 699, 700, 701, 702, 703, 704, 705, 706, 707, 708, 709, 710, 711, 712, 713, 714, 715, 716, 717, 718, 719, 720, 721, 722, 723, 724, 725, 726, 727, 728, 729, 730, 731, 732, 733, 734, 735, 736, 737, 738, 739, 740, 741, 742, 743, 744, 745, 746, 747, 748, 749, 750, 751, 752, 753, 754, 755, 756, 757, 758, 759, 760, 761, 762, 763, 764, 765, 766, 767, 768, 769, 770, 771, 772, 773, 774, 775, 776, 777, 778, 779, 780, 781, 782, 783, 784, 785, 786, 787, 788, 789, 790, 791, 792, 793, 794, 795, 796, 797, 798, 799, 800, 801, 802, 803, 804, 805, 806, 807, 808, 809, 810, 811, 812, 813, 814, 815, 816, 817, 818, 819, 820, 821, 822, 823, 824, 825, 826, 827, 828, 829, 830, 831, 832, 833, 834, 835, 836, 837, 838, 839, 840,"

" 841, 842, 843, 844, 845, 846, 847, 848, 849, 850, 851, 852, 853, 854, 855, 856, 857, 858, 859, 860, 861, 862, 863, 864, 865, 866, 867, 868, 869, 870, 871, 872, 873, 874, 875, 876, 877, 878, 879, 880, 881, 882, 883, 884, 885, 886, 887, 888, 889, 890, 891, 892, 893, 894, 895, 896, 897, 898, 899, 900, 901, 902, 903, 904, 905, 906, 907, 908, 909, 910, 911, 912, 913, 914, 915, 916, 917, 918, 919, 920, 921, 922, 923, 924, 925, 926, 927, 928, 929, 930, 931, 932, 933, 934, 935, 936, 937, 938, 939, 940, 941, 942, 943, 944, 945, 946, 947, 948, 949, 950, 951, 952, 953, 954, 955, 956, 957, 958, 959, 960, 961, 962, 963, 964, 965, 966, 967, 968, 969, 970, 971, 972, 973, 974, 975, 976, 977, 978, 979, 980, 981, 982, 983, 984, 985, 986, 987, 988, 989, 990, 991, 992, 993, 994, 995, 996, 997, 998, 999]'
"

That is, the string is split before calling the filter functions. The problem seems relative to this, even the output in the comint buffer in my situation does not contain any \n.

I want to work with the whole single output in my filter function instead of the split ones. How can I achieve this?

Thanks in advance.

| improve this question | |
  • (elisp) Filter Functions: "The output to the filter may come in chunks of any size [...] one way [...] is to insert the received text into a temporary buffer, which can then be searched." – npostavs Apr 28 '17 at 16:13
  • @npostavs Thanks for your information. Can you tell me how can I collect all these chunks in a filter function or how I can detect when the last chunk was send to the filter function? – stackunderflow Apr 28 '17 at 16:24
  • 1
    @npostavs Nope, since the original single string may also contain \n. I can't figure out what's the mechanism of collecting all the parts of the original string. – stackunderflow Apr 28 '17 at 16:55
  • 1
    @npostavs I originally want to turn the output of str(list(range(1000))) into [0, ..., 999]. Since the string is split and sent to the filter function, the result become five parts "'[0, ... 225, 22","6, ... 430, 4","31, ... 635, ","636, ..., 840,"," 841, ... 999]'" and comint concat all these back into a single string and insert into the buffer. That is, even I want to truncate the string manually, the final string could still be very long if there are may chunks and I can only truncate each chunk of the string. – stackunderflow Apr 28 '17 at 17:02
  • 1
    @npostavs I use python for scientific computation and since the data is quite big quite often I accidentally enter a list name of somewhat python doesn't output prettily but a single long line which freezes my comint buffer and Emacs. I have to kill emacs and restart. With a proper pretty print option (e.g. a threshold of number of elements) of package like numpy, the display of numpy object (e.g. ndarray) is automatically truncated while the native python list not. – stackunderflow Apr 29 '17 at 2:43
0

You can check for the prompt to detect the end of output:

(defvar python-output nil)
(defun python-shell-filter-long-lines (string)
  (if (not (string-match comint-prompt-regexp string))
      (progn (push string python-output)
             "")
    (let ((out (mapconcat #'identity (nreverse python-output) "")))
      (setq python-output nil)
      (if (> (length out) 20)
          (concat (substring out 0 10) "..."
                  (substring string (- (match-beginning 0) 10)))
        (concat out string)))))

(add-hook 'comint-preoutput-filter-functions #'python-shell-filter-long-lines)
| improve this answer | |
  • Thanks for your answer. But I can't see the prompt of inferior shell after I run run-python with this filter. – stackunderflow Apr 29 '17 at 4:56
  • I had a bug for the short non-chunked output case. – npostavs Apr 29 '17 at 12:33
2

I slightly modify @npostavs 's answer for my own use. I'll accept his answer and give him the credit. 

(defvar python-shell-output-chunks nil)
(defun python-shell-filter-long-lines (string)
  (push string python-shell-output-chunks)
  (if (not (string-match comint-prompt-regexp string))
      ""
    (let* ((out (mapconcat #'identity (nreverse python-shell-output-chunks) ""))
           (split-str (split-string out "\n"))
           (max-len (* 2 (window-width)))
           (disp-left (round (* (/ 1.0 3) (window-width))))
           (disp-right disp-left)
           (truncated (mapconcat
                       (lambda (x)
                         (if (> (length x) max-len)
                             (concat (substring x 0 disp-left) " ... (*TRUNCATED*) ... " (substring x (- disp-right)))
                           x))
                       split-str "\n")))
      (setq python-shell-output-chunks nil)
      truncated)))

The snippet truncates every long line in a single output in my inferior python(ipython) shell. For example

enter image description here

instead of the original one which almost freeze and kill emacs. I accidentally input a variable which inferior-python-mode does not print it friendly quite often especially when debugging in pdb. Hope this help.

enter image description here

| improve this answer | |
  • I suggest you accept your own answer, since that's the one that completely solves your problem. Mentioning that my answer helped you a bit is enough credit. – npostavs Apr 29 '17 at 12:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.